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1.0 PROTOTYPING A MULTI -BACKEND DATABASE SYSTEM (MDBS) 



Four approaches to the running of a database management system have been 
proposed in the literature: 

(1) Running the database management system along with all other software 
on a single general-purpose computer, known as the host . 

(2) Running the database management system on a second general-purpose 
computer system, known as the backend . This is known as the 
single-backend software approach • 

(3) Developing a special-purpose database machine with specially designed 
hardware to perform the database management functions* This is known 
as the database machine or hardware backend approach* 

(4) Running the database management system on multiple general-purpose 
computers. This is known as the multi-backend software approach. 

Database management systems built using the first approach have some 
limitations, e.g., as the database grows and the rate of requests to database 
system increases, the host computer performance decreases. Database manage- 
ment systems built using the second approach have the same limitation, i.e., 
the performance of the single-backend system also decreases. Thus, overall 
performance of the host and backend will be degraded. The third approach may 
be promising, but not until the cost-effectiveness of this approach is demon- 
strated . 

The fourth approach configures the backends in a parallel way for per- 
formance improvement. It also allows growth in the database and increase in 
the request rate without performance degradation and software complexity. 
This approach requires the development of an innovative software design which 
allows the addition of more backends of the same type and the replication of 
the software on the new backends without major system interruption. Thus, it 
does not require the development of any new hardware, but only the develop- 
ment of a new and replicable software architecture and a new and parallel 
hardware configuration. This report is the second in a series [Kerr 82] which 
describes the development of a multi-backend database system known as MDBS as 
a prototype for experimenting with the fourth approach. 
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1 . 1 Logical Description of MDBS 

In this section, we give a brief review of MDBS, Full details on the 
design and analysis of MDBS can be found in [Hsia81a] and [Hsia81b] . The 
first report in this series [Kerr82] gives a more detailed overview. 

1.1.1 The MDBS Hardware and Software Organization 

An overview of MDBS hardware organization is shown in Figure 1. The 
controller and the backends are connected by a broadcast bus. The controller 
will broadcast each request to all backends at the same time over this bus. 
Furthermore, there will be minimal broadcasting from one backend to the other 
backends . 

Each backend is given a number of dedicated disk drives. The data from 
each file is distributed across all the backends. Each backend will then 
process the data from its own disk drives. Because each file is spread 
across all the backends, all backends can now execute the same request in 
parallel. Request execution at a backend is handled by having a queue of re- 
quests at the backend. When a backend finishes executing one request it can 
start executing the next request. In view of the execution mode, MDBS is a 
multiple-instruction-and-multiple-data-stream (MIMD) organizaton . 

The data model chosen for the system is the attribute-based data model 
[Hsia70] . In MDBS the database consists of files of records. Each record is 
a collection of keywords, optionally followed by a record body. A keyword is 
made of an attribute-value pair such as <SALARY ,12000 > where $12,000 is the 
value of the attribute SALARY. A record body is a string of characters not 
used by MDBS for search purposes. An example of a record without a record 
body is shown below. 

( <FILE, Employees <EMPLOYEE_NAME , Smith> , <CITY,Columbus> , 

<SALARY , 1 2000 > , <SERVICE,10> ) 

The first attribute-value pairs in all records of a file are the same. In 
particular, the attribute is FILE and the value is the file name . For exam- 
ple, the above record is from the Employee file. When dealing with the re- 
cords of the same file, we frequently omit the first attribute-value pair, 
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Figure 1. The MDBS Hardware Organization 
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i.e., the file name, for illustration. 

For performance reasons, records are logically grouped into clusters 
based on the attribute values and attribute value ranges in the records. 
These values and value ranges are called descriptors . For example, one clus- 
ter might contain records for those employed in Columbus, making at least 
$20,001 but not more than $25,000 and with at least 11 but not more than 15 
years of service. Thus records of this cluster are grouped by the following 
three descriptors: 

(CITY=Columbus) , ( 20001 =<SALARY=<25000 ) , (11=<SERVICE=<15) . 

MDBS performs its operations by clusters. Thus finding records of employees 
in Columbus making between $21,000 and $22,000 per year and with 12 to 13 
years experience would require the retrieval of records in the cluster just 
described. Other retrieval requests such as to find records of employees in 
Columbus making between $21,000 and $28,000 and with 12 to 13 years experi- 
ence might require additional retrieval of records from other clusters than 
the one identified above. 

In order to allow efficient processing of requests, records in a cluster 
are spread across all the backends. Thus each backend needs to search only 
its portion of the cluster. Given a user request, there must be a way, of 
course, first to determine which clusters to search and then to determine the 
location of records in a given cluster. To perform this task, MDBS utilizes 
available descriptor information. For example, given the previous request 
for finding employees where 

( CITY=Co lumbus ) and (21000=<SALARY=<28000) and (12=<SERVICE=<13 ) 

MDBS first determines that two clusters must be searched. These are the 
clusters identified by the two sets of descriptors: 

{ ( CITY=Co lumbus ) , ( 20001 =<SALARY= <25000 ) , (11=<SERVICE=<15) } 

{ ( CITY=Co lumbus ) , ( 25001 =<SALARY=<30000 ) , ( 11=<SERVICE=<15) } 

After the clusters are identified, MDBS must then determine the disk ad- 
dresses of the clusters at each backend. Finally MDBS will cause each back- 
end to retrieve from its disks the records so addressed. 

The execution phases of a retrieval request are summarized in Figure 2. 
Descriptor search determines the descriptors that correspond to the request. 
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In our example, there are four descriptors corresponding to the request; 
namely, 

( CITY=Co lumbus ) , ( 2000 1=<SALARY= <25000) , 

(25001= s <SALARY= <30000) , (11=KSERVICE=<15) . 

In order to save space and to save processing time each descriptor is identi- 
fied by a descriptor id . For example. 



Descriptor 


Descriptor Id 


( CITY=Co lumbus ) 


D15 


( 20001=<SALARY=<25000 ) 


D125 


( 25001=<SALA£Y=<30000 ) 


D126 


( 11=<SERVICE=<15 ) 


D250 



Thus the output of the descriptor search phase is the Boolean expression 
descriptor ids 

D15 and (D125 or D126) and D250 (1) 

corresponding to 



f( 



( 20001 =<SALARY=<25000) 



( CITY^Co lumbus ) and < 



l( 



or 



J25001=<SALARY=<30000) 
which identifies two clusters. 



and (11-<SERVICE-<15) 



of 



The next phase, cluster search , must take the Boolean expression in (1) 
and actually determine the corresponding clusters. As with descriptors, 
clusters are also identified by ids, known as cluster ids , for example 



Descriptor Ids 


Cluster Id 


D15, D125, D250 
D15, D126, D250 


C17 

C22 



The final two phases, see Figure 2, are address generation (to find the 
disk addresses, e.g., A3 546 and A3 547, corresponding to each cluster id, 
e.g., C17) and record processing (to retrieve the actual records so addressed 
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and extract the fields required). 

Descriptor search, cluster search and address generation together form 
the major portion of directory management . 

Concurrent processing of requests is facilitated in MDBS. Executing one 
request at a time at a backend will frequently leave the backend's CPU idle 
while waiting for its disk to access records. Since the MDBS hardware organ- 
ization provides multiple disk drives per backend, it is possible for a back- 
end to support concurrent processing of requests from different users. 
However, a mechanism to control concurrent access to data must then be pro- 
vided. Because all directory management is based on the concept of clusters, 
it is also logical to design a concurrency control mechanism based on clus- 
ters. Thus, the mechanism used in MDBS is centered on the concept of clus- 
ters. In particular, the concurrency control mechanism will lock clusters to 
prevent conflicting access to the same clustered data. 

The general method used by MDBS in processing a retrieval request is 
described and summarized in Figure 3. In the next section, we will show how 
this processing is carried out among the controller and the backends. 



1.1.2 Distribution of Request Execution Among Controller and Backends 

In the previous section, we mentioned how the database was distributed 
across the backends. However, we did not discuss the placement of directory 
data and the distribution of the directory processing in directory manage- 
ment. In order to minimize the time for directory management and to facili- 
tate record update, the directory data is duplicated at all backends. On the 
other hand, the directory processing is not duplicated at each backend. For 
instance, the descriptor search phase is divided among the backends. Each 
backend must find a different subset of descriptor ids. It then broadcasts 
its results to all the other backends. 

In Figure 4, we summarize how directory management is performed at a 
backend. A retrieval request is received from the controller. Then the 
backend performs a descriptor search on its portion of the request and broad- 
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casts the resulting descriptor ids to the other backends. After the descrip- 
tor ids from all other backends have been received, cluster search is used to 
determine the clusters* Finally, address generation determines the local 
disk addresses for records at that backend. 

The backend can do more than just retrieve all the records in a cluster. 
First, it can select those and only those records that satisfy the request. 
For example, the request to find records of employees in Columbus earning 
more than $20,000 but not more than $28,000 and with more than 10 but not 
more than 15 years experience, requires selecting records from two clusters. 
Those clusters are identified by 

(CITY=Columbus ) and ( 20001 =<SALARY=<25000) and (11=<SERVICE=<15) 

and 

( CITY=Co lumbus ) and ( 25001 =<SALARY=<30000) and (11=<SERVICE=<15) . 

All the records will be selected from the first cluster, but only records 
with SALARY~<$28 ,000 will be selected from the second cluster. 

Often, not all the data in a record is needed to respond to a request. 
In this example, only the names of the employees might be required. Thus the 
appropriate values must be extracted from the record. The other values may 
be discarded. Although not shown in this example, MDBS can perform various 
types of aggregate operations on a set of values instead of just returning 
the raw values. An example would be to find the average salary of employees 
who live in Columbus. Thus after selecting the appropriate records and ex- 
tracting the salary values, MDBS would compute the average. The steps of 
record processing are summarized in Figure 5. 

Referring to Figure 6, the execution of a user request can now be sum- 
marized as follows. The user submits a request to the host which transmits 
that request, in an internal form, to the controller of MDBS. The controller 
parses the request and then broadcasts it to the backends. Each backend de- 
termines its portion of the descriptor ids and broadcasts the results to the 
other backends. Each backend also determines the clusters that must be 
searched and the corresponding local disk addresses. Then the appropriate 
records are selected, values extracted and results sent back to the controll- 
er. When the controller has received the results from all the backends, it 
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Figure 6. Modes of MDBS Operations 
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performs any aggregate operation required and then forwards the final results 
to the host for return to the user. 



1 .2 The Imp 1 ement a t ion Strategy - What and Why? 

It seems only reasonable to develop most systems in stages. For proto- 
type systems such an approach seems even more important. Thus we have 
planned to develop several versions of MDBS. We chose to begin with an im- 
plemention of a very simple system. 



1.2.1 Version I - A Very Simple System: Single Mini Without Concurrency 

Control and With Simplified Directory Management 

We started the implementation effort with a system which was intended to 
be as simple as possible. The aim was to get something running so that we 
could gain some experience with both the MDBS design and our new computer 
systems. Thus we had chosen to simplify the design as much as possible. 
MDBS-I, which is in the final stages of implementation, executes only a sin- 
gle request at a time. It runs on a single computer. There is no distinc- 
tion made about the slave and master. In other words, there is no separate 
controller. Directory management is simplified by storing all directory data 
in the main memory. There are no concurrent execution of requests. Since 
the whole system runs as a single operating system process, the interface 
with the operating system is minimized. 



1.2.2 Version II - A Simple System: Single Mini With Concurrency Control 

The second version, whose details have been designed and the implementa- 
tion effort is under way, will allow concurrent execution of requests, but 
will still be restricted to a single mini. We plan to use the services of 
our operating system to facilitate this concurrent processing. Thus we will 
use the capability of creating independent concurrent processes which commun- 
icate among themselves. These processes will execute in parallel so that 
MDBS-II will be able to execute requests in parallel. This version will 
allow us to gain experience with the problem of multiple processes and the 
problem of concurrency control. 
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1.2.3 Version III - The First "Real" System : Multiple Minis With 

Concurrency Control 

After MDBS-II is working, we will transfer the system to our real envi- 
ronment including a controller (i.e., VAX 11/780) and several backends (PDP 
ll/44s). This transfer should be fairly easy, since the major changes re- 
quired will be to replace communications between processes in one computer by 
communications between processes running on different computers. This ver- 
sion will allow us to see how the intercomputer communication overhead is 
going to affect system performance. This system, MDBS-Ill, will still not be 
sufficient for a full MDBS, since it has a very simplified directory manage- 
ment subsystem. However, it will allow us to begin preliminary testing of 
the MDBS design. 



1.2.4 Version IV - The Real System With "Good" Directory Management 

This version, whose design details are being proposed, will include a 
fully implemented directory management subsystem utilizing the secondary mem- 
ories. it will be a complete prototype system, except for the lack of access 
control features. This system, MDBS-IV, will be the one on which we will try 
to validate the simulation studies used in the development of the original 
design. 



1.2.5 Version V - The Full System With All the Designed Features Included 

The final version will incorporate access control in the backends and a 
friendly user-interface in the controller or host computer. 



1 .3 An Overview of the MDBS Implementation 

In this section, we give an overview of the implementation effort to 
date. The recent implementations are described in more detail in later 
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chapters. Details on our earlier implementation effort can be found in 
[Kerr82] . 



1.3.1 A Top-level View of MDBS 

The MDBS is viewed in terms of controller functions and backends func- 
tions (see Figure 7). In the following two sections, we describe the func- 
tions performed in the controller and the backends, respectively. Then we 
will describe the process of request execution for four types of request: 
delete, insert, retrieve and update. 

There are, however, some essential functions which are not included in 
either of these divisions. Among these are system generation, system 
startup/shutdown, and other system utilities such as database load, file gen- 
eration and database reorganization. These functions will generally be ini- 
tiated in the minicomputer which serves as the MDBS controller. They are 
not, however, a logical part of the major functions of the controller. 



1.3.2 Functions of the Controller 

The MDBS controller consists of three categories of functions: request 

preparation, insert information generation and post processing (see Figure 7 
again) . The request preparation functions are those which must be performed 
before a request or a transaction can be broadcasted to the backends. For 
example, each request must be parsed and checked for syntax errors before it 
can be broadcasted to the backends. The insert information generation 
functions are those which must be performed during the processing of an in- 
sert request to furnish additional information required by the backends. For 
example, a backend should be selected for storing the record being inserted 
into the secondary storage of the backend. The post processing functions are 
those which must be performed after replies are returned from the backends, 
but before the results of a request or a transaction are forwarded to the 
host machine. For example, the results for a request returned by each back- 
end should be collected. After receiving the results from each backend, the 
response to the request can be sent to the host machine. 
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Figure 7. 



The MDBS Structure 
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We note that there are no concurrency control functions in the controll- 
er. Since user requests are carried out by the backends, there is no need 
for concurrency control in the controller. The controller must only associ- 
ate sequence numbers with the user requests. 



1.3.3 Functions of each Backend 

Each backend in MDBS consists of three categories of functions: 
directory management, record processing and concurrency control (also in Fig- 
ure 7). The directory management functions perform descriptor search, clus- 
ter search, address generation and directory table maintenance. For example, 
these functions find the ids of descriptors corresponding to a set of predi- 
cates (keywords), determine the cluster id corresponding to a set of descrip- 
tors and determine the secondary storage addresses of the records in a clus- 
ter. The record processing functions perform record storage, record retriev- 
al, record selection and attribute value extraction of the retrieved records. 
For example, these functions store records into the secondary storage, 
retrieve records from the secondary storage and select the records that 
satisfy a query from a set of records. The concurrency control functions 
perform operations which ensure that the concurrent and interleaved execution 
of user requests will keep the database consistent. For example, these fuc- 
tions schedule a user request for execution based on the set of clusters 
needed by the request. In this chapter, we do not consider concurrent and 
interleaved execution of user requests. The concurrency control mechanism is 
described in Chapter 4. 



1.3.4 Request Execution in MDBS 

In this section, we describe briefly the sequence of actions taken by 
MDBS in executing insert requests and non-insert requests (delete, retrieve 
and update). The sequence of actions is described in terms of flow of data 
and in terms of the functions categorized above. The sequenece of actions 
taken by MDBS in executing each of the four types of request: insert, de- 
lete, retrieve and update is described in more detail in the later chapters. 
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(A) Sequence of Actions for an Insert Request 

The sequence of actions for an insert request is shown in Figure 8. 
Some flow of data is common to all types of request, shown as dotted lines in 
the figure. Thus, we first describe these common data flows. The arrow 
entering Request Preparation indicates that a request or a transaction is the 
input to this module. The input comes from the host machine. Request Pre- 
paration sends the number of requests in a transaction to Post Processing. 
The number of requests in a transaction is used by Post Processing to deter- 
mine whether processing of the transaction is complete. Request Preparation 
also sends a request (transaction) along with error messages to Post Process- 
ing if the request (transaction) has syntax errors. Post Processing collects 
all the results related to a request (transaction) and sends the results to 
the host machine. The arrow leaving Directory Management indicates that the 
descriptor ids found by a backend are sent to the other backends. The arrow 
entering Directory Management indicates that the descriptor ids found by the 
other backends are sent to this backend. 

We now describe the flow of data specific to insert requests, shown as 
solid lines in Figure 8. After receiving, parsing and formatting a request, 
Request Preparation sends the formatted request to Directory Management in 
the backends. We recall that the record part of the request consists of many 
keywords and each backend performs the descriptor search for a different set 
of keywords in the record. Thus, Directory Management at a backend finds the 
ids of descriptors corresponding to the set of keywords to be processed at 
the backend and broadcasts the ids to the other backends. After receiving 
the descriptor ids sent by the other backends, Directory Management deter- 
mines the cluster id, if any, of the cluster to which the record belongs. It 
then sends the cluster id to Insert Information Generation in the controller. 
Insert Information Generation determines the backend at which the record is 
to be inserted and broadcasts a message to Directory Management in the back- 
ends. The backends that are not to insert the record discard the record. 
Directory Management in the backend that is to insert the record determines 
the secondary storage address for inserting the record. That address and the 
formatted request are then passed to Record Processing. Record Processing 
stores the record into the secondary storage and sends a completion signal to 
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Figure 8. Sequence of Actions for an Insert Request 
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Post Processing in the controller. Post Processing then sends a completion 
signal to the host machine. 

(B) Sequences of Actions for Non-insert Requests 

The sequences of actions for non- insert requests are all similar. Thus, 
we describe the sequence of actions only for a retrieve request in this sec- 
tion. This is shown in Figure 9. (Here, we assume that the retrieve request 
was not caused by an update request. Details on retrieve requests caused by 
update requests are given in Chapter 2.) 

Request Preparation, after receiving, parsing and formatting a request, 
sends the formatted request to Directory Management in the backends and the 
aggregate operators, if any, in the request to Post Processing in the con- 
troller. We recall that the query part of the request consists of many pred- 
icates and each backend performs the descriptor search for a different set of 
predicates in the query. Thus, Directory Management at a backend finds the 
ids of descriptors corresponding to the set of predicates to be processed at 

the backend and broadcasts the ids to the other backends. After receiving 
* 

the descriptor ids sent by the other backends, Directory Management deter- 
mines the cluster ids. Finally, it determines the secondary storage ad- 
dresses of the records in the clusters so identified and sends the record ad- 
dresses and the formatted request to Record Processing. Record Processing 
fetches the records from the secondary storage and selects the records that 
satisfy the query. It then extracts the values from the selected records. 
If aggregation is not needed, Record Processing sends the extracted values to 
Post Processing in the controller. Post Processing collects all the results 
related to the request and sends the results to the host machine. 

If some aggregations are to be applied. Record Processing, after select- 
ing the records and extracting the values, applies the aggregate operations 
on the set of values. It then sends the results to Post Processing in the 
controller. The partial results from all the backends are collected in Post 
Processing. Post Processing performs the aggregate operations on the partial 
results and sends the results to the host machine. 
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1.3.5 The Role of the Connnunication Interface 

Let us now describe the boxes labeled Communication Interface in Fig- 
ures 8 and 9. They provide the mechanism for communications between two 
functions in two different computers. There is a communication interface in 
each computer, i.e., the controller and the backends, since certain functions 
in each computer must communicate with certain functions in the other com- 
puters . 



1 .4 The Organization of the Rest of the Report 

We describe in detail the MDBS implementation in the rest of this re- 
port. In Chapter 2, we give a functional description of MDBS. The controll- 
er and the backends functions are described in detail in Chapters 3 and 4, 
respectively. A method for testing MDBS is described in Chapter 5. Finally 
in Chapter 6, we summerize our software engineering experience. 
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2.0 A FUNCTIONAL DESCRIPTION OF MDBS 



As described in the previous chapter and depicted in Figure 7, MDBS is 
viewed in terms of controller functions and backend functions. In this 
chapter, we describe the functions of the controller and backends in detail. 
We also describe in detail the process of request execution for four types of 
request: delete, insert, retrieve and update. 



2.1 Functions of the Controller 

The MDBS controller functions are considered in three categories: 
request preparation, insert information generation and post processing. (As 
described in Chapter 1, there are no concurrency control functions in the 
controller.) In the following, we describe the functions of each of the three 
categories. We will not discuss the system functions such as system 
startup/shutdown in this section. We will describe a package for testing 
MDBS in Chapter 5. Other details on the system functions such as database 
load and file generation can be found in [Kerr82] . 



2.1.1 The Request Preparation Functions 

These are the functions which must be perfomed before a request or a 
transaction can be broadcasted to the backends. The names of the functions 
are: Parser and Request Composer. 

(A) The Parser Function 

This function parses the requests and checks for syntax errors. Input 
to Parser comes from the host machine. The input is either a request or a 
transaction. If the input request (transaction) is parsed correctly, then 
the parsed request (parsed transaction) is passed to Request Composer. If 
the input request (transaction) contains syntax errors, Parser returns the 
request (transaction) along with error messages to Reply Monitor. MDBS does 
not execute a transaction unless all the requests in the transaction are 
parsed correctly, i.e., a transaction is rejected if one or more requests 
contain syntax errors. 
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For retrieve requests with aggregate operators, Parser sends the type of 
aggregate operators (AVG, MAX, MIN, SUM, COUNT) to Aggregate Post Operation 
where the specific aggregate operations are to be performed on the partial 
results to be returned by the backends. 



When the input to Parser is a transaction, Parser passes the number of 
requests in the transaction to Reply Monitor. The number of requests in a 
transaction is used by Reply Monitor to determine whether the processing of 
the transaction is complete. 

(B) The Request Composer Function 



Before describing this function, let us review the update requests in 
MDBS. The syntax of an update request is: 

UPDATE Query Modifier 

where the modifier specifies the kinds of modification that need to be done 
on records that satisfy the query. The modifier may be one of the following 
five types: 

Type-0 : <attribute=constant> 

Type-I : <attribute=f (attributed 

Type-II : <attribute=f (attributel) > 

Type-Ill : <attribute=f (attributel) of Query> 

Type-IV : <attribute=f (attributel) of Pointer> 



Let a record whose attribute is being modified be referred to as the 
record being modified . Then, a type-0 modifier sets the new value of the at- 
tribute being modified to a constant. A type-I modifier sets the new value 
of the attribute being modified to be some function of its old value in the 
record being modified. A type-II modifier sets the new value of the attri- 
bute being modified to be some function of some other attribute value in 
the record being modified. A type-III modifier sets the new value of the at- 
tribute being modified to be some function of some other attribute value in 
another record uniquely identified by the query in the modifier. Finally, a 
type-IV modifier sets the new value of the attribute being modified to be 
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some function of some other attribute value in another record identified by 
the pointer in the modifier. 

An example of a type-0 modifier is: 

<SALARY=5G000> 

This sets the salary in all the records being modified to 50000. 

An example of a type-I modifier is: 

< S ALARY =1 . 1 *SALARY> 

This raises the salary in all the records being modified by 10%. 

An example of a type-II modifier is: 

<HONTHSAL=YEARS AL/ 1 2 > 

This sets the monthly salary in all the records being modified to be a 
twelfth of their own yearly salaries. 

An example of a type-III modifier is: 

<SALARY= SALARY of (FILE-Wife) and (NAME=Tara)> . 

This causes the following actions to be taken by MDBS. Using the query 
"(FILE=Wif e) and (NAME=Tara) " , a record is retrieved. Then, the SALARY value 
of that record is obtained. This value is used for the salary in all the re- 
cords being modified. 

An example of a type-IV modifier is: 

<SALARY= SALARY of 2000 > 

which modifies the salary in all the records being modified to that of the 
record stored in location 2000. In order to use this type of modifier, the 
user must have previously issued a retrieve request which had WITH POINTER 
option. We note that, in order to execute an update request containing a 
type-III or type-IV modifier, a record must first be retrieved by MDBS on the 
basis of a user-provided query or pointer. We now describe the Request Com- 
poser function. 

This function transforms a parsed request into the form required for 
processing at the backends. Request Composer receives each parsed request 
(parsed transaction) from Parser. For all requests except updates with 
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type-III or type-IV modifier, Request Composer formats the request and sends 
it to the backends for processing. For update requests with type-III or 
type-IV modifier, Request Composer first generates a retrieve request. It 
then saves all the information necessary to generate an update request with 
type-0 modifier when the value from the retrieve request is received. When 
the value is received from a backend, the update request with type-0 modifier 
will be generated and sent to the backends. 

Processing an update request may cause one or more updated records to 
change cluster. When this occurs, the old records should be marked for dele- 
tion and the updated records should be inserted. Request Composer initiates 
the actions required for the insertion of the updated records that change 
cluster . 



2.1.2 The Insert Information Generation Functions 

These are the functions which must be performed during the processing of 
an insert request to furnish additional information required by the backends. 
The names of the functions are: Backend Selector, Cluster Id Generator and 
Descriptor Id Generator. 

(A) The Backend Selector Function 

When processing an insert request, Backend Selector determines the back- 
end at which the record is to be inserted. The backend selection is based on 
the criterion that the records in each cluster should be distributed among 
the backends. (As described in Chapter 1, the records in each cluster are 
spread across the backends to allow the records in the cluster to be accessed 
in parallel.) 

(B) The Cluster Id Generator Function 

In order to save storage and time, each cluster is identified by a clus- 
ter id, instead of being identified by a set of descriptors which character- 
ize the cluster. Cluster Id Generator produces a new cluster id for a new 
cluster • 



PAGE 27 



(C) The Descriptor Id Generator Function 

To further save storage and time, each descriptor is also identified by 
a descriptor id, instead of being identified by an attribute and its attri- 
bute value (attribute value ranges). Descriptor Id Generator produces a new 
descriptor id for a new descriptor. 



2.1.3 The Post Processing Functions 

Before the results of a request or a transaction are forwarded to the 
host machine, these functions must be performed on the replies returned by 
the backends. The names of the functions are: Aggregate Post Operation and 
Reply Monitor. 

(A) The Aggregate Post Operation Function 

When there is an aggregate operator in a retrieve request, each backend 
performs the aggregate operation on those records in that backend satisfying 
the query. The partial aggregate results are sent to Aggregate Post Opera- 
tion by the backends. Parser sends the type of aggregate operator (AVG, MAX, 
MIN, SUM, COUNT) to Aggregate Post Operation where the partial results are 
received from the backends and are combined to give the final result of the 
specific aggregate operation. The results are then forwarded to Reply Moni- 
tor. 

(B) The Reply Monitor Function 

This function collects all the results for a request or a transaction, 
and forwards them to the host machine. As described earlier, Parser sends 
the number of requests in a transaction to Reply Monitor. Reply Monitor uses 
this number to determine whether the processing of the transaction is com- 
plete . 
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2.2 Functions of each Backend 

Each backend in MDBS consists of three categories of functions: 
directory management, record processing and concurrency control (see Figure 7 
again), (As in Chapter 1 , we do not consider concurrent and interleaved exe- 
cution of user requests in this chapter. We describe the concurrency con- 
trol mechanism in Chapter 4.) In the following sections, we describe the 
functions of each of the first two categories, i.e., directory management and 
record processing. 



2.2.1 The Directory Management Functions 

These functions perform directory operations such as cluster determina- 
tion, address generation and directory table maintenance. The names of the 
functions are: Descriptor Search, Cluster Search and Address Generation. 

(A) The Descriptor Search Function 

This function determines the descriptor ids of the descriptors that 
satisfy the predicates (keywords) in a query (record). Input to Descriptor 
Search comes from Request Composer in the controller, in the form of a for- 
matted request. As described in detail in [Hsia81a] , if there are N backends 
processing a query (record) with X predicates (keywords), then each backend 
performs the descriptor search on X/N predicates (keywords) and broadcasts 
the descriptor ids to the other backends. 

(B) The Cluster Search Function 

This function determines either the cluster id of the cluster to which a 
record belongs (for an insert request) or the cluster ids of the clusters 
whose records satisfy a query (for a non-insert request). Input to Cluster 
Search are the descriptor ids found by Descriptor Search in all the backends. 
For insert requests. Cluster Search passes the cluster id, if any, to Backend 
Selector in the controller. For non- insert requests, the cluster ids are 
passed to Address Generation. 



PAGE 29 



(C) The Address Generation Function 

This function determines either the secondary storage address for stor- 
ing a record when processing an insert request or the addresses of all the 
records in a set of clusters when processing a non-insert request. For in- 
sert requests, Backend Selector in the controller determines which backend is 
to insert the record. When a backend is selected. Address Generation in that 
backend determines the secondary storage address for record insertion. That 
address and the formatted request are then passed to Physical Data Operation. 

For non-insert requests, Cluster Search passes the cluster ids to Ad- 
dress Generation. Address Generation finds the addresses of the records in 
these clusters and passes the addreses and the formatted request to Physical 
Data Operation. 



2.2.2 The Record Processing Functions 

These functions perform operations such as record selection and field 
extraction of the retrieved records. The names of the functions are: 
Physical Data Operation and Aggregate Operation. 

(A) The Physical Data Operation Function 

Input to this function comes from Address Generation. The input is a 
set of secondary storage addresses and a formatted request. Physical Data 
Operation performs different actions depending on the type of the request. 
For an insert request, Physical Data Operation stores the record being in- 
serted into the secondary storage. 

For a non-insert, i.e., delete, retrieve or update, Physical Data Opera- 
tion fetches the records from the secondary storage and selects the records 
that satisfy the query in the request. It then performs the intended opera- 
tion on the basis of the type of the non-insert request. For delete re- 
quests, Physical Data Operation marks the selected records for deletion. 

For retrieve requests, Physical Data Operation extracts the values from 
the selected records and passes the values either to Aggregate Operation, if 
an aggregation is to be applied, or to Reply Monitor, if aggregation is not 
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needed. For retrieve requests caused by update requests with type-III or 
type-IV modifier, Physical Data Operation sends the results to Request Com- 
poser in the controller. The results will be used in the controller to form 
update requests with type-0 modifier from the update requests with type-III 
or type-IV modifier. 

For update requests, Physical Data Operation updates the selected 
records and returns to the secondary storage those updated records that have 
not changed cluster. If one or more records change cluster, Physical Data 
Operation marks the old records for deletion and sends the records that have 
changed cluster to Request Composer in the controller. Request Composer ini- 
tiates the actions required for the insertion of these records into their new 
clusters . 

(B) The Aggregate Operation Function 

This function performs the partial aggregate operations in retrieve re- 
quests. Input to Aggregate Operation comes from Physical Data Operation in 
the form of a set of values and the aggregate operators to be applied. 
Aggregate Operation applies the aggregate operations on the set of values and 
passes the results to Aggregate Post Operation in the controller. 



2.3 Request Execution in MDBS 

In this section, we describe in detail the sequence of actions taken by 
MDBS in executing each of the four types of request: insert, delete, 
retrieve and update. As in Chapter 1, the sequence of actions is described 
in terms of flow of data and in terms of functions presented earlier. 



2.3.1 Sequence of Actions for Insert Requests 

The sequence of actions for an insert request is shown in Figure 10. As 
in Chapter 1, we first describe the flow of data common to all types of re- 
quest, shown as dotted lines in Figure 10. The arrow entering Parser indi- 
cates that a request or a transaction is the input to this function. The 
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Figure 10. Sequence of Actions for an Insert Request 
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input comes from the host machine . Parser sends the number of requests in a 
transaction to Reply Monitor, The number of requests in a transaction is 
used by Reply Monitor to determine whether the processing of the transaction 
is complete. Parser also sends a request (transaction) along with error mes- 
sages to Reply Monitor if the request (transaction) has syntax errors. Reply 
Monitor collects all the results related to a request (transaction) and sends 
them to the host machine. The arrow leaving Descriptor Search indicates that 
the descriptor ids found by a backend are sent to the other backends. The 
arrow entering Cluster Search indicates that the descriptor ids found by the 
other backends are sent to this backend. 

We now describe the flow of data specific to insert requests, shown as 
solid lines in Figure 10. Parser, after receiving and parsing a request, 
sends the parsed request to Request Composer. After transforming the parsed 
request into the form required for processing at the backends, Request Com- 
poser sends the formatted request to Descriptor Search in the backends. We 
recall that the record part of the request consists of many keywords and each 
backend performs the descriptor search for a different set of keywords in the 
record. Thus, Descriptor Search at a backend finds the ids of descriptors 
corresponding to the set of keywords to be processed at the backend, 
broadcasts the ids to the other backends and forwards them to Cluster Search. 
Cluster Search determines the cluster id, if any, of the cluster to which the 
record belongs. It then sends the cluster id to Backend Selector in the con- 
troller. Backend Selector determines the backend at which the record is to 
be inserted and broadcasts a message to Address Generation in the backends. 
The backends that are not to insert the record discard the record. Address 
Generation in the backend that is to insert the record determines the secon- 
dary storage address for storing the record. That address and the formatted 
request are then passed to Physical Data Operation. Physical Data Operation 
stores the record into the secondary storage and sends a completion signal to 
Reply Monitor in the controller. Reply Monitor then sends a completion sig- 
nal to the host machine. 



2.3.2 Sequence of Actions for Delete Requests 



The sequence of actions for a delete request is shown in Figure 11. 
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Figure 11. Sequence of Actions for a Delete F.equest 
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Parser, after receiving and parsing a request, sends the parsed request to 
Request Composer. After transforming the parsed request into the form re- 
quired for processing at the backends, Request Composer sends the formatted 
request to Descriptor Search in the backends. Descriptor Search at a backend 
finds the ids of descriptors corresponding to the set of predicates to be 
processed at the backend, broadcasts them to the other backends and forwards 
them to Cluster Search. Cluster Search determines the cluster ids and gives 
them to Address Generation. Address Generation determines the secondary sto- 
rage addresses of the records in these clusters and sends the record ad- 
dresses and the formatted request to Physical Data Operation. Physical Data 
Operation fetches the records from the secondary storage. It then selects 
the records that satisfy the query, marks the selected records for deletion, 
returns them to the secondary storage and sends a completion signal to Reply 
Monitor in the controller. 



2.3.3 Sequence of Actions for Retrieve Requests 

The sequence of actions for a retrieve request is shown in Figure 12. 
Parser, after receiving and parsing a request, sends the parsed request to 
Request Composer and the aggregate operators, if any, in the request to Ag- 
gregate Post Operation. The sequence of actions taken by Request Composer, 
Descriptor Search, Cluster Search, Address Generation and Physical Data Oper- 
ation (up to the selection of the records that satisfy the query) is the same 
as the other non-insert request, i.e., delete. Thus, we do not repeat it 
here . 



If the retrieve request was not caused by an update request, Physical 
Data Operation extracts the values from the selected records. If aggregation 
is not needed, Physical Data Operation sends the extracted values to Reply 
Monitor in the controller. If some aggregations are to be applied, Physical 
Data Operation passes the extracted values along with the aggregate operators 
to Aggregate Operation. Aggregate Operation applies the aggregate operations 
on the set of values and sends the results to Aggregate Post Operation in the 
controller. The partial aggregate results from all the backends are collect- 
ed in Aggregate Post Operation. Aggregate Post Operation performs the aggre- 
gate operations on the partial results. The results are then forwarded to 



PAGE 35 




Figure 12. Sequence of Actions for a Retrieve Request 
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Reply Monitor. Reply Monitor collects all the results related to the request 
and sends the results to the host machine. 

If the retrieve request was caused by an update request, Physical Data 
Operation sends the result, if any, to Request Composer in the controller. 
(The results will be used in the controller to form an update request with 
type-0 modifier from the update request with type-III or type-IV modifier.) 



2.3.4 Sequence of Actions for Update Requests 

The sequence of actions for an update request is shown in Figure 13. 
Parser, after receiving and parsing a request, sends the parsed request to 
Request Composer. The sequence of actions will be different depending on the 
type of modifier in the update request. We first describe the case where the 
modifier is not type-III or type-IV. In this case, the sequence of actions 
taken by Request Composer, Descriptor Search, Cluster Search, Address Genera- 
tion and Physical Data Operation (up to the selection of the records that 
satisfy the query) is the same as the other non-insert request, i.e., delete. 
Thus, we do not repeat it here. 

Physical Data Operation updates the selected records. It then uses Des- 
criptor Search to determine which updated records have changed cluster. 
Physical Data Operation stores those updated records that have not changed 
cluster into the secondary storage. It will then send a completion signal to 
Reply Monitor in the controller if no updated record has changed cluster. 

If one or more updated records change cluster, Physical Data Operation 
marks the old records for deletion and sends the records that have changed 
cluster to Request Composer in the controller. Request Composer initiates 
the actions required for the insertion of these records into their new clus- 
ters. After these records are inserted, the original update request is com- 
plete. 



If the modifier in the update request is type-III or type-IV, Request 
Composer in the controller first generates a retrieve request. It then saves 
all the information necessary to generate an update request with type-0 mod- 
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Figure 13, Sequence of Actions for an Update Request 
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ifier when the value from the retrieve request is received* When Request 
Composer receives the value from Physical Data Operation, it generates the 
update request with type-0 modifier and sends it to the backends. After this 
new update request is executed to completion, the original update request is 
complete . 



2.4 Process Structure of MDBS 

Most operating systems provide mechanisms for allowing concurrent execu- 
tion of different processes. These mechanisms include primitives for commun- 
ication and synchronization among processes. Process communication and syn- 
chronization primitives of the operating system are the basic system primi- 
tives that MDBS-II utilizes for concurrent execution of multiple requests as 
well as concurrent control of common resources. 



2.4.1 Two Alternative Process Structures for Implementing MDBS 

Process and synchronization primitives provided by the operating systems 
may be characterized as either message-oriented or procedure-oriented, de- 
pending on how they implement the notion of process and synchronization 
[Laue79l. We could use either approach for implementing MDBS. 

Using a message-oriented approach, there would be a fixed number of 
processes (one process per MDBS activity). Directory management, for exam- 
ple, may be implemented as a process. Synchronization of directory manage- 
ment activities may be implemented by passing messages among processes. 
There would be a relatively limited amount of direct sharing of data in the 
memory among processes. Processes for each activity would be created when 
MDBS is started up. They would be deleted only when MDBS is shut down. 

Using a procedure-oriented approach, there would be a varying number of 
processes (one process per user). Synchronization of user activities may be 
implemented by direct sharing and locking of common data in the main memory. 
Processes would be rapidly created and deleted. 
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2.4.2 The Choice of Message-oriented Approach to Implement MDBS 

The functional composition of MDBS described in the previous sections 
allows either approach, message-oriented or procedure-oriented, to be used 
for implementing MDBS. However, we have chosen to use message-oriented ap- 
proach for the first implementation of MDBS-II. In this section, we give the 
rationale behind our choice. 

There are two major problems associated with the procedure-oriented ap- 
proach [Ston81]: 

(1) Process switch overhead - When a process must be put to wait, a pro- 
cess switch is necessary in order to run another process. Process 
switching is costly because the information related to the blocked 
process must be saved and the processor scheduler must conduct con- 
siderable work to choose the next process to run. Since the proce- 
dure-oriented approach causes more process switches than the 
message-oriented approach, the process switch overhead is higher in 
this approach. 

(2) Critical sections - Some processes have critical sections in which 
holds on locks are placed. If the processor scheduler deschedules a 
process while it is in its critical section holding some locks over 
some resource, all other processes will be queued up behind the 
locked resource. Thus, the database system performance will be de- 
graded. 

The real-time operating system, RSX11, being used in MDBS facilitates 
message passing. It also allows a process to receive messages from multiple 
proceses. Because of the aforementioned two problems with the 
procedure-oriented approach and because of the environment provided by RSX11, 
we have decided to use the message-oriented approach. 
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3.0 AN IMPLEMENTATION OF THE CONTROLLER FUNCTIONS 



3 .1 Design and Imp I ement a t ion Goals for the Controller 

The primary goal in designing and implementing the controller subsystem 
of MDBS is to alleviate the controller limitation problem , i.e., to limit the 
amount of work that the controller must perform. The choice of a solution to 
the controller limitation problem is prompted by another design and implemen- 
tation goal for MDBS, that of minimizing communication among the backends and 
between the backends and the controller . Without increasing workload and ex- 
cessive communication, the throughput of MDBS will continue to increase as 
additional backends are added. 

The controller limitation problem occurs in RDBM [Auer80] , a relational 
database machine, where a general-purpose minicomputer is used to control the 
different hardware components of the system and to pre-process user requests. 
Request pre-processing includes a detailed analysis of the request to deter- 
mine the pages in the secondary memory to be accessed. The speed of the min- 
icomputer is therefore a limiting factor to the throughput of RDBM. Consider 
a simplified example where preprocessing a user request requires 10 seconds 
of CPU time at the minicomputer, regardless of the number of backends in the 
system. The throughput rate of RDBM is limited to 6 requests per minute. 

Another view of the controller limitation problem is from the perspec- 
tive of response time. The total response time of the system may be viewed 
as the sum of controller execution time and backend execution time. Adding 
more backends can decrease the backend execution time, but controller execu- 
tion time remains constant. So in order to minimize request execution time, 
we must also minimize controller execution time. 

Our controller design is based on the principle that the major portion 
of the MDBS workload should be distributed among the backends. In adherence 
to this principle, the controller is conceptually simple and includes primar- 
ily those functions which cannot be performed by the multiple backends. 
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3.2 The Concept of "Traffic Unit " 

Input to MDBS originates from a user working at some host computer. The 
host computer translates the user's instructions into the MDBS Data Manipula- 
tion Language (DML) and transmits the translated requests to MDBS. This 
transmission or "traffic" may take two forms: it may be a single request , or 
it may be a transaction . Recall that in MDBS terminology, a transaction is 
defined to be a pre-specif ied set of requests which the user may use repeat- 
edly. 



In order to generalize the description of input to MDBS, we introduce 
the concept of a traffic unit . A traffic unit may be a single request or a 
transaction. The identification of a traffic unit is important to the host, 
since it must return to the user all output from MDBS associated with the 
traffic unit. The recognition of a traffic unit as a single request or as a 
transaction is also important to MDBS, since transactions must be processed 
in a manner which preserves the consistency of the database. Since the 
traffic unit is recognized in the host, we assume that the host will associ- 
ate with each traffic unit currently in the system a unique identifier, which 
we call the traffic id. 



3.3 The Structure of the Controller 

The MDBS Controller is implemented in three functional categories: 
Request Preparation, Insert Information Generation, and Post Processing. The 
Request Preparation functions include those which must be performed before a 
request or transaction can be broadcasted to the backends. The Insert Infor- 
mation Generation functions include those which must be performed during the 
processing of an insert request to furnish additional information required by 
the backends. The Post Processing functions include those which must be per- 
formed after replies are returned from the backends, but before results of a 
request or a transaction are forwarded to the host machine. These three ca- 
tegories of functions have been described in Chapter 2 of this report. In 
this chapter, we present details of the implementations of these functions. 
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3.3*1 The Request Preparation Functions 

The Request Preparation functions include the Parser and Request Com- 
poser* The Parser function parses the requests and checks for syntax errors* 
The Request Composer function transforms a parsed request into the form re- 
quired for processing at the backends* These functions have been described 
in Section 2.1.1. Here, we emphasize the implementation. 



(A) The Parser Function 

Parser does both lexical and syntactic analyses of the MDBS DML state- 
ments. Input to Parser is in terms of a traffic unit, i.e. either a single 
request or a group of requests which constitute a transaction. As described 
in Section 2.1.1, the various outputs of the parser are the error messages 
and aggregation operators to the Post Processing functions, and correctly 
parsed requests to the Request Composer function. 

The lexical analyzer was built using the LEX program available with the 
UNIX operating system. LEX [Lesk79] is a lexical-analyzer generator which 
can be used to generate programs in C. The input to LEX is a specification 
of the tokens of the language (i.e., the tokens of the MDBS DML statements) 
in regular expression form, and subroutines which specify the actions to be 
taken upon recognition of the tokens. LEX generates a program in the C 
language. This program includes a representation of a deterministic 
finite-state automaton generated from the regular expressions of the source, 
an interpreter which directs the control flow, and the subroutines from the 
source. The lexical analyzer produced by LEX is easily interfaced with the 
parser generated by YACC. 

The parser was built using the YACC program available with UNIX. YACC 
[John79] , "Yet Another Compiler-Compiler ", was used to generate a parser 
which calls the LEX -generated lexical analyzer for tokens, and organizes the 
tokens according to rules of a grammar. When a rule is recognized, some 
specified action is taken. The input to YACC is a specification which in- 
cludes declarations of token names, the rewriting rules of the grammar, and 
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action programs. YACC produces a C program, i.e., the parser, according to 
the specification. The parser operates like a finite-state automaton with a 
stack. The top-of-stack represents the current token. The parser also has 
access to the next token, called the lookahead token. Using this simple me- 
chanism, the parser can determine whether input DML statements are syntacti- 
cally correct. For a detailed explanation of YACC, see [ John79] . 



(B) The Request Composer Function 

The Request Composer receives parsed requests from the Parser, and 
transforms them into the form required for processing at the backends. 
Recall from Section 2.1.1 that update requests with type-III and type-IV mod- 
ifiers require Request Composer to generate a retrieve request, and a subse- 
quent update request with a type-0 modifier. Request Composer also initates 
the actions required for the insertion of updated records that have changed 
cluster. Since the implementation of Request Composer is straightforward, it 
will not be described further. 



3.3.2 The Insert Information Generation Functions 

Insert Information Generation consists of three functions: Backend Se- 
lector, Cluster Id Generator, and Descriptor Id Generator. When processing 
an insert request, the Backend Selector function determines the backend at 
which the record is to be inserted. The Cluster Id Generator function pro- 
duces new cluster ids for new clusters. The Descriptor Id Generator function 
produces new descriptor ids for new descriptors. The functions are described 
in Section 2.1.2. Before we describe any implementation details, let us re- 
view the types of descriptors which are defined in MDBS. 

As described in Chapter 1, records in the database are clustered on the 
basis of attribute values and attribute value ranges called descriptors. 
There are three types of descriptors: type-A, type-B, and type-C. A type-A 
descriptor defines an inclusive range of values. Each type-A descriptor is a 
conjunction of a less-than-or-equal-to predicate and a greater-than- 
or-equal-to predicate. An example of a type-A descriptor is: 

((SALARY >= 2,000) and (SALARY =< 10,000)) 
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A type-B descriptor defines a single value. Each type-B descriptor consists 
of an equality predicate. An example of a type-B descriptor is: 

(POSITION = Professor) 

A type-C descriptor designates an attribute name as a type-C attribute. As 
records are inserted into the database, a single-valued descriptor is created 
for each unique value associated with the type-C attribute. These descrip- 
tors, which are identical to type-B descriptors, are referred to as type-C 
sub-descriptors . 

Type-A and type-B descriptors, type-C attributes and type-C subde- 
scriptors are created at database-load time. No additional descriptors can 
be defined after the database is loaded. Type-C sub-descriptors, however, 
will be created dynamically as new records are inserted into the database. 



(A) The Backend Selector Function 

In order to conform to the data placement strategy described in 
[Hsia81a] , the controller must determine the backend at which the record is 
to be inserted. This is the function of Backend Selector. 

The information required for selecting the backend is maintained in the 
cluster-id-to-next-backend table (CINBT) . There is an entry in the table for 
each cluster. Each entry contains the number of the next backend into which 
records are to be inserted, and the remaining track capacity at that backend. 
The CINBT is created at database load time. CINBT is implemented as a data 
abstraction. The operations on this data abstraction, insert, find and up- 
date, will be invoked by Backend Selector in accessing CINBT. 

At the end of the descriptor search phase in processing an insert re- 
quest, each backend will send to Insert Information Generation the cluster id 
for the record to be inserted. Since the cluster-definition table (CDT) is 
not replicated, backends at which no records of a cluster are stored will not 
find a cluster id for that cluster. There is also the case where the record 
being inserted has caused a new type-C sub-descriptor to be generated; in 
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this case, no backends will return a cluster id. When Backend Selector de- 
termines that all backends have responded, it will proceed to select the 
backend at which the record is to be inserted. 



(B) The Cluster Id Generator Function 

In order to save storage and time, each cluster is identified by a clus- 
ter id. The Cluster Id Generator generates a new cluster id when there is a 
new cluster. There are two cases which require a new cluster id. These 
cases are described in (A) above. 



(C) The Descriptor Id Generator Function 

When an insert request contains a record with a type-C attribute and the 
value associated with that attribute does not appear in a type-C 
sub-descriptor, a new type-C sub-descriptor will be created. The assignment 
of descriptor ids is handled by the Controller to prevent coincidental crea- 
tion of different descriptor ids by the backends for the same descriptor. If 
two simultaneous insert requests requiring the creation of the same type-C 
sub-descriptor were processed by the backends independently, different des- 
criptor ids would be assigned for the same descriptor. In MDBS, descriptors 
must have unique ids. 

Descriptor Id Generator will generate a new descriptor id when request- 
ed, and broadcast descriptor id and descriptor to all backends. Descriptor 
Id Generator will retain a list of all descriptors to which it has assigned 
descriptor ids. This list will be consulted each time a request for a new 
descriptor id is received in order to prevent coincidental creation of dif- 
ferent descriptor ids for the same descriptor. The list will also be purged 
periodically. 



3.3.3 The Post Processing Functions 



The Post Processing functions include Aggregate Post Operation and Reply 
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Monitor. The Aggregate Post Operation function performs the final aggregate 
operation on partial aggregate results returned from the backends. The Reply 
Monitor function collects all the results for a request or transaction, and 
forwards them to the host machine. These functions are described in Section 
2.1.3. No further implementation details are presented here. 



3 *4 The Process Structure of the Controller 

Since a message-oriented approach to concurrency control is being used, 
we must choose a process structure for the Controller. There are several ob- 
vious choices. 

First, all of the functions of the Controller can be combined into one 
process. This alternative is unattractive because it limits the Controller 
to one function at a time. A greater degree of concurrency can be obtained 
by using multiple processes and the multiprogramming facilities of the under- 
lying operating system. A second alternative is to create a process for each 
of the seven functions of the Controller. While this does allow a high de- 
gree of concurrency, it 'is unattractive because of the message-passing 
overhead. 

A third alternative is to use a smaller number of processes to facili- 
tate concurrency, while keeping the message-passing overhead at an acceptable 
level. A good candidate organization is one which parallels the categories 
of functions which we have described above. There are three processes: the 
Request Preparation process, the Insert Information Generation process, and 
the Post Processing process. Look again at Figures 10, 11, 12 and 13 from 
Chapter 2. These figures show the flow of data between Controller and Back- 
ends functions for insert, delete, retrieve, and update requests. Requests 
flow from the host through the Request Preparation process to the Backends, 
and from the Backends through the Post Processing process to the host. In 
the case of insert and update requests, the Insert Information Generation 
process will be exchanging data with Directory Managment in the Backends. 
Notice that the only interprocess communication in the Controller will be 
between the Request Preparation and Post Processing processes. This is the 
organization we adopt for the process structure of the MDBS Controller. 
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4.0 AN IMPLEMENTATION OF BACKEND FUNCTIONS 

As discussed in Section 3.1, a basic design goal of MDBS is to assign as 
much work as possible to the backends in order to alleviate the controller 
limitation problem. Consequently, the backends functions are more complex 
than those of the controller. The functions of the backends fall into three 
categories: Directory Management, Record Processing and Concurrency Control. 

The Directory Management functions perform directory operations such as 
cluster determination, address generation, and directory table maintenance. 
According to the incremental development strategy described in Chapter 1, two 
versions of Directory Management will be developed. A simplified Directory 
Management, where all directory information is stored in main memory, is des- 
cribed in [Kerr82] . This simplified Directory Management will be used in the 
first three versions of MDBS (MDBS-IV and V will employ a secondary- 
memory-based directory management). 

The Record Processing functions perform operations such as record selec- 
tion and attribute value extraction. The design of these functions is des- 
cribed in detail in Section 4.1. 

A second design goal for MDBS is that the software should support con- 
current execution of requests in the backends in order to maximize system 
throughput. The cluster-based concurrency control functions, described in 
[Hsia81b] , will be implemented in Version II of MDBS. In Section 4.2, we 
present a preliminary design of the Concurrency Control functions. 



4.1 The Record Processing Functions 

The Record Processing functions are: Physical Data Operation and Aggre- 
gate Operation^ The Physical Data Operation function includes a control sub- 
function and a subfunction for each type of request. The Retrieve Processing 
Subfunction, the Insert Processing Subfunction, the Delete Processing Sub- 
function, and the Update Processing Subfunction are invoked by the Control 
Subfunction according to the type of request being processed. The Aggregate 
Operation function includes subfunctions which accumulate partial aggregate 
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results for a request when an aggregate operation is specified for an attri- 
bute in the target-list. The Aggregate Operation Subfunctions are invoked 
as required by the Retrieve Processing subfunction. 

The Retrieve Processing subfunction, the Insert Processing subfunction, 
the Delete Processing subfunction, and the Update Processing subfunction are 
described in detail in the sections which follow. Here, we give a general 
description of the Control subfunction. 

The input to Record Processing comes from the Directory Management func- 
tions. Input data includes: 

(1) a request; 

(2) a set of physical (disk) addresses of the tracks which contain 
data relevant to the request; 

(3) in the case of an insert request, an indicator which is used to 
determine whether the record is to be placed on a new track. 

The specific form of the output varies with the type of request; a general 
description of the output is a signal to the Controller that execution of the 
request is completed, and the results of execution. 

The sequence of events is as follows: 

Step 1: Input is received from Directory Management. 

Step 2: The proper subfunction is invoked according to the request 

type. 

Step 3: The results are sent to the Controller. 

Step 4: A completion signal is sent to the Controller. 

The results of a retrieve or an update request may include many records. 
Thus, the results are buffered independently via a data abstraction, the 
Block_ Buff er_Abs tract ion, which is also described below. 



4.1.1 The Block_Buf f er_Abs traction 

In MDBS, a cluster may correspond to more than one physical track of 
data on the disk. Therefore, for one cluster, there may be more than one 
physical address in the set of addresses furnished to Record Processing by 
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Directory Management. Data are accessed from or to the disk track by track. 
So, a fixed-length buffer can be used for input data. 

The amount of output data varies from request to request. This implies 
that, given a fixed-length output buffer, the Record Processing functions 
must include logic to empty the output buffer when it is filled during execu- 
tion of a request. In order to simplify the Record Processing functions, a 
data abstraction is used to implement a virtual variable-length output 
buffer. This technique has two advantages. First, the Record Processing 
functions will not need to include logic to monitor the state of the output 
buffer. Second, all the logic required to use the communication interface 
for sending results to the Controller can be localized in the code of the 
data abstraction. 

The Block_Buff er_Abs tract ion furnishes a data object, the Result_Buff er , 
and a set of operations. The operations include a function to reserve a 
buffer, a function to stuff data into a buffer, and a function to flush a 
partially filled buffer. The actual data structure used by the abstraction 
is a fixed-length buffer. However, the stuff operation includes logic to 

9 

empty filled buffers. It appears to the user that the output buffer is as 
large as required. 



4.1.2 The Retrieve Processing Subfunction 

A retrieve request has the form: 

RETRIEVE Query Target-List [BY clause] [WITH pointer] 

The purpose of the Retrieve Processing is to fetch the clusters of relevant 
data from the disk, to select from the clusters of relevant data the records 
satisfying the query, and to output the results according to the target-list 
and the optional BY and WITH clauses. 

The algorithm is as follows: 

Step 1: Reserve a result buffer. 

Step 2: For each address in the set of track addresses furnished by 

Directory Management, fetch the track from the disk into the 
track buffer in the main memory. 
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Step 3: Examine the records in the track buffer one-by-one. If a re- 

cord is marked for deletion, disregard it. If a record does 
not satisfy the query of the request, disregard it. If a re- 
cord satisfies the query of the request, extract the values 
for the attribute names in the target-list of the request; 
if an aggregate operation is specified for an attribute on 
the target-list, invoke the appropriate aggregation subfunc- 
tion with the appropriate value. Stuff results from extrac- 
tion and/or aggregation into the result buffer. Repeat for 
each record in the track buffer. 

Step 4: Repeat steps 2 and 3 until the set of track addresses is ex- 

hausted. 

Step 5: Flush the result buffer. 

If the optional WITH clause is included, a pointer or physical address 
of the record is stuffed into the result buffer for each record^ The option- 
al BY clause is used in conjunction with an aggregate operator, as explained 
in the next section. 



4.1.3 The Aggregation Subfunctions 

MDBS supports five aggregate operations on attributes in the target-list 
of retrieve requests. These are AVG, SUM, COUNT, MAX and MIN. An example of 
a target-list is: 

(DEPT, AVG( SALARY)) 

No aggregate operator is specified for the attribute DEPT; the values of 
DEPT will be retrieved from all records identified by the query. The aggre- 
gate operator AVG will be applied to the values of SALARY retrieved from all 
records identified by the query. Thus, the average salary will be obtained. 

An optional BY clause may be used with an aggregate operator. Assume 
that we wish to find the average salary of employees in each department. 
This can be achieved by using a retrieve request with the target-list 
(AVG( SALARY)) and the clause BY DEPT. 



The aggregation subfunctions are invoked by the Retrieve Processing sub- 



PAGE 51 



function as required. For AVG, a sum of values and a count is accumulated. 
For SUM, a sum of values is accumulated. For COUNT, a count of values is ac- 
cumulated. For MAX and MIN, the maximum and minimum elements are selected. 



4.1.4 The Insert Processing Subfunction 

The insert request has the form: 

INSERT Record 

The purpose of the Insert Processing subfunction is to insert the record in 
the request into a cluster. The record may be added to a partially-filled 
track of data, or may be inserted as the first record of a newly allocated 
track. The input to Record Processing for an insert request includes a 
new-track indicator. Since only one record is being inserted into one track 
of one cluster. Directory Management will furnish only one track address. 

The algorithm for the Insert Processing subfunction is very simple : 

Step 1: If the new-track indicator is off (meaning that the record is 

to be added to a track that already contains other records 
from the cluster) , then fetch the track from the disk into 
the track buffer. If the new-track indicator is on, then in- 
itialize the track buffer (no data are fetched from the 
disk) . 

Step 2: Insert the record in the request into the track buffer. 

Step 3: Store the track buffer on the disk. 



4.1.5 The Update Processing Subfunction 

The update request has the form: 

UPDATE Query Modifier 

The modifier in an update request specifies the new value to be taken by the 
attribute being modified. The modifier may be one of the types described 
below. 

Type-0 : <attribute = constant> 

Type-I : <attribute = f (attributed 

Type-II : <attribute = f (attributel) > 
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Type-III : <attribute = f (attributel) of Query> 

Type-IV : <attribute = f (attributel) of Pointer> 

The Update Processing subfunction handles requests with modifiers of 
type-0, I or II, An update request with the modifier of type-III or type-IV 
is decomposed by the Controller into a retrieve request followed by an update 
request of type-0* 

The main function of Update Processing subfunction is to select records 
satisfying the query and to update the value of the attribute specified by 
the modifier* When a type-0 modifier is specified, the new value is the con- 
stant from the modifier* When a type-I modifier is specified, the new value 
is a function of the old value* When a type-II modifier is specified, the 
new value is a function of the value of some other attribute in the record* 

If the attribute being updated is a directory attribute, the updated re- 
cord may change cluster* This occurs when the updated value does not corres- 
pond to the same descriptors as the value before update. In this case, the 
set of descriptors which can be derived from the record is not the same as 
the set of descriptors which defines the current cluster. If the updated re- 
cord changes cluster, then the original record is marked for deletion and the 
updated record is sent to Request Composer in the Controller. Request Com- 
poser will generate an insert request for the updated record. If the updated 
record does not change cluster, then it is simply rewritten in the same clus- 
ter. 



The algorithm is as follows: 

Step 1: Reserve a result buffer. 

Step 2: For each address in the set of track addresses furnished by 

Directory Management, fetch the track from the disk into the 
track buffer in the main memory. 

Step 3: Examine the records in the track buffer one-by-one. If a re- 

cord is marked for deletion, disregard it. If a record does 
not satisfy the query of the request, disregard it. If a re- 
cord satisfies the query of the request, compute the new 
value according to the modifier and update the record in the 
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track buffer. Check the updated record to determine whether 
it changes cluster. If it does, then the updated record is 
added to the result buffer and marked for deletion from the 
track buffer. 

Step 4: After all of the records in the track buffer have been exam- 

ined, store the track buffer back to the disk. 

Step 5: Repeat Step 2 through Step 4 until the set of track addresses 

is exhausted. 

Step 6: Flush the result buffer and send the results to Request Com- 

poser in the controller. 



4.1.6 The Delete Processing Subfunction 

The delete request has the form: 

DELETE Query 

The purpose of the Delete Processing subfunction is to delete all the records 
satisfying the query. Records are not physically deleted from the database. 
They are instead marked for deletion. Records will be physically deleted 
only when the database is reorganized. 

The algorithm is as follows: 

Step 1: For each address in the set of track addresses furnished by 

Directory Management, fetch the track from the disk into the 
track buffer in the main memory. 

Step 2: Examine the records in the track buffer one-by-one. If a re- 

cord is marked for deletion, disregard it. If a record does 
not satisfy the query of the request, disregard it. If a re- 
cord satisfies the query of the request, set a deletion flag 
in the record. 

Step 3: Repeat Step 1 and Step 2 until the set of track addresses is 

exhausted. 

Step 4: Store the track buffer on the disk. 
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4.2 Concurrency Control 

\ 

In the previous sections, all consideration of the concurrent execution 
of requests has been omitted. However as was mentioned in Chapter 1, the 
backends must allow concurrent execution of requests in order to assure effi- 
cient processing of the requests. This section will first present a brief 
review of the concurrency control mechanism which was described in detail in 
[Hsia81b], Then it will provide more details concerning the implementation. 

Concurrency control is a mechanism by which we will insure the consis- 
tency of the database while allowing concurrent execution of multiple re- 
quests. To insure the consistency of the data, locks are utilized. These 
locks are administered at the cluster level (i.e., individual clusters are 
locked). There are five phases of execution of a request in the presence of 
access control and concurrency control. First, directory management deter- 
mines the clusters needed by the request. Second, cluster access control de- 
termines the authorized clusters. Third, concurrency control determines when 
all clusters needed by the request are available. Fourth, address generation 
determines the record addresses. Finally, record processing actually exe- 
cutes the request. 



4.2.1 Two Types of Consistency 

The MDBS Concurrency Control mechanism differs from others in the types 
of locks as well as in their utilization. The mechanism distinguishes the 
four types of requests (Update, Retrieve, Insert, and Delete) and utilizes a 
different lock mode for each type. 

There are two types of consistencies which must be assured. The first 
type of consistency is called inter-consistency . One example of the type of 
problem we are concerned with is two concurrent updates of a record, which 
might result in the loss of one of the updates. This problem must be consi- 
dered in both single and multiple backend systems. To preserve 
inter-consistency, non-concurrent execution must be assured among requests 
which may have different results when executed simultaneously. Requests 
which may execute concurrently are called compatible requests. The compati- 
bility of two requests depends on the mode of access, e.g., two retrieve re- 
quests are compatible whereas two update requests are not. When considering 
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a new request, if the mode of the new request is not compatible with that of 
one of the earlier requests which is executing, then the execution of the new 
request must be delayed* Thus the MDBS concurrency mechanism locks clusters 
so that only compatible requests can be using a cluster at the same time* 

As just described, requests are executed at the backends in the order 
they are received from the controller. Sometimes for performance reasons, 
however, it may be desirable to permute the order of execution of two re- 
quests that are not compatible. For example, suppose a sequence of three re- 
quests R1 , R2 and R3 are received and Rl requires cluster Cl, R 2 requires 
clusters Cl and C2, while R3 requires cluster C2. In a single backend sys- 
tem, it would be possible to permute the execution of requests R2 and R3 , al- 
lowing R3 to execute concurrently with Rl since Rl and R3 require different 
clusters. In order to permute the order of execution of requests in a 
multi-backend system, however, a mechanism must be found to assure that all 
backends execute the requests in the same order. Otherwise inconsistent re- 
sults can again occur. Thus in a multi-backend system it is also necessary 
to assure intra-cons is tenc v » i.e., requests that are not compatible must exe- 
cute in the same order at all backends. 

A general mechanism to allow the permutation of requests that are not 
compatible would be complex because it would require communication among all 
the backends. However a simple mechanism can be found that will handle the 
special case involving an insert request. The actual insertion of a new re- 
cord is performed at only one backend. It is not distributed across all the 
backends. Therefore, if the backends are allowed to permute a non-insert re- 
quest and an insert request, then the effective order of execution of the re- 
quests at all the backends is the order used by the backend which actually 
performs the insertion. In general, two requests that are not compatible are 
called permutable if they do not have to be executed in the same order at all 
the backends. Thus we see that an insert request and a non-insert request 
are permutable and we can assure intra-consistency if we permute the execu- 
tion order only of permutable requests. 

The compatibility and permutability of requests can be summarized as 



follows : 
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Delete 
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Insert 


P 


C 
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P 


N 


N 


Retrieve 


N 


P 


N 


C 



C = Compatible 
P - Permutable 

N = Not permutable and not compatible 



This table shows that two delete requests, or two insert requests or two 
retrieve requests are compatible because they can be executed concurrently 
without the possibility of inconsistency developing. It also shows that an 
insert request can be permuted with a non-insert request, i.e., a delete, an 
update or a retrieve. As was explained above, this permutability of an in- 
sert request with a non-insert request is due to the fact that the actual in- 
sert occurs at only one backend. Only the* delete, update or retrieve is ac- 
tually performed at all the backends. Thus the effect is the same as it 
would have been if all the backends executed the requests in the order used 
by the backend performing the insert. 

The concurrency control mechanism described in [Hsia81b] assures that 
requests which are not permutable or compatible are executed, without over- 
lap, in the order received by the controller. Permutable requests can, 
however, be executed in any order so long as they do not overlap at the same 
backend. So as to keep track of all the requests, each backend maintains a 
queue of requests for each cluster, in the order in which the controller re- 
ceived the requests. Thus no later request can execute before an earlier re- 
quest that is not permutable has been executed. In addition no permutable 
requests can execute concurrently, although the order of execution can be 
modified. On the other hand compatible requests can execute together. 
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4.2*2 Two Categories of Locks 

Unfortunately, allowing the permutation of requests means that a new 
problem may now occur, the problem of starvation . It may be possible to per- 
mute one request indefinitely. Thus that request will never be allowed to 
execute. In order to prevent starvation, we introduce two categories of 
locks: "to-be-used" and "being-used" . As soon as a request reaches a back- 
end, it locks the clusters it needs in the "to-be-used" category. Before it 
can execute, it must convert the locks to the "being-used" category. Only 
requests which are locking a cluster in the "to-be-used" category are allowed 
to be permuted. Thus starvation can be prevented. Details of how this 
conversion of a lock from "to-be-used" to "being-used" and how this mechanism 
allows the permutation of requests while preventing starvation are discussed 
below. First however we must relate transactions to concurrency control. 



4.2.3 The Notion of Transaction 

A user may wish to treat a set of requests as a transaction . Such a set 
of requests is known by the user to preserve the consistency of the database 
if executed alone on a database system running on a single computer. Users 
may want execution of a transaction to begin before all the requests in the 
transaction have been provided to MDBS. In this case, we call the transac- 
tion incompletely-specif ied . Unfortunately, because all clusters required by 
the incompletely-specified transaction cannot be determined before execution 
of the transaction is to ‘begin, there is no algorithm which allows the use of 
incompletely-specified transactions without sometimes having to backup one of 
two transactions which have been executing concurrently. Thus in MDBS, we 
have chosen to restrict transactions to those that are pre-specif ied , i.e., 
all the requests in a transaction must be submitted to MDBS at the same time 
and before execution of any of the requests in the transaction begins. Then 
MDBS must convert all locks to the "being-used" category before execution of 
the transaction can begin. Locks can then be released as requests in the 
transaction finish execution. 

In the previous section, when we discussed compatible and permutable re- 
quests, we assumed the requests were not part of a transaction. We must now 
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reexamine these concepts in the context of transactions. Since two compati- 
ble requests have no affect on each other, we can still allow their concur- 
rent execution even when one is part of a transaction. On the other hand, 
the order of execution of two permutable requests does affect the result. 
Thus the whole transaction should be permuted, rather than one of its re- 
quests. Because of the complexity of permuting a whole transaction, we have 
chosen to permute only requests that are not part of a transaction. 



4.2.4 Concurrency Control Using a Message-Oriented Approach 

The concurrency control mechanism was described in [Hsia81b] using a 
procedure-oriented approach. Thus there was to be a lock table shared 
by all users. In addition, transactions were deactivated when a needed clus- 
ter was locked by other requests and were activated when the needed cluster 
became available. 

This basic mechanism must now be transformed to reflect a 
message-oriented approach. In this approach, as described earlier, there is 
a concurrency control process . This process receives messages from the di- 
rectory management process (a request to be executed) and from the record 
processing process (a report that a request has completed execution). When 
the concurrency control process determines that a request is ready for execu- 
tion it forwards the request to record processing. The "shared lock-table" 
evident in the procedure-oriented approach now appears as a table internal to 
the concurrency control process. This table, called the 
cluster-to-traf f ic-unit table (CTUT) , is described in Section (B) below. The 
concept of "deactivating" a transaction is replaced by having concurrency 
control hold the request in a queue until it can be forwarded to record pro- 
cessing for execution. The algorithms for concurrency control are described 
in Section (D) below. 

(A) The Process Structure in the Backends 

Once a message-oriented approach has been selected, it is necessary to 
break up the functions of each backend into processes. The most obvious 
choice would be to have one process per function, i.e., five processes cor- 
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responding to descriptor search, cluster search, concurrency control, address 
generation and record processing, respectively. (The sixth function, cluster 
access control is omitted because it is not included in our initial implemen- 
tation.) However, since there is added overhead for each interprocess mes- 
sage, it is desirable to reduce the number of processes. One easy way to do 
this is to combine descriptor search and cluster search into a single direc- 
tory management process. Address generation must take place after concurren- 
cy control, since records may be added to a cluster while a request is wait- 
ing to lock the cluster. Thus, address generation cannot be included in a 
directory management process. However, it could be combined with either con- 
currency control or record processing. For the purposes of discussing con- 
currency control, it is easiest to assume that address generation is not part 
of concurrency control. Thus the function of concurrency control is to sche- 
dule the execution of requests based on the clusters that are required as de- 
termined by directory management. 

(B) Cluster-To-Traff ic-Unit Table (CTUT) 

As was described earlier, information about the locks held on each clus- 
ter is stored in the CTUT. This table contains a queue for each cluster. 
Each cluster queue contains an entry for each of the requests requiring that 
cluster. Each entry contains an identifier for the request (the traffic-unit 
and the request-number), the MODE of access required (delete, insert, 
retrieve or update), and the CATEGORY of lock held ("to-be-used" or 
"being -used") . A sample CTUT with four clusters is shown in Figure 14. This 
table contains entries for five single requests and one transaction consist- 
ing of two requests. 

(C) Traffic-Unit-To-C luster Table (TUCT) 

In a procedure-oriented implementation there is a process associated 
with each user and this process keeps track of how many locks are still to be 
acquired before a transaction can be executed. However, in a 
message-oriented implementation, of course, there is no such process for a 
user. Thus this information must be maintained in a different way. The con- 
currency control process stores this information in a traff ic-unit- to-c luster 
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table (TUCT) , which it can then use to determine the status of any 
traffic-unit. This table is essentially an inverse of the CTUT, It is a 
reference, by traffic-unit, of which clusters are required for each request 
of the traffic-unit. In addition, this table keeps track of how many re- 
quests of the transaction have not yet been sent to record processing for ex- 
ecution. Figure 15 shows the TUCT corresponding to the CTUT shown in Fig- 
ure 14. 



(D) The Processing of Concurrency Control Information 

The concurrency control process receives messages from directory manage- 
ment and from record processing. A message from directory management con- 
sists of a new request to be executed and a list of clusters required by that 
request. A message from record processing means that execution of a request 
has been completed. Concurrency control must send messages to record pro- 
cessing notifying it to begin execution of a request. 

In order to handle these messages, concurrency control must perform 
three basic functions. When a new traffic-unit is received from directory 
management, an initialization must be performed locking all the required 
clusters in the "to-be-used" category. When concurrency control receives a 
message from record processing that execution of a request has been complet- 
ed, then concurrency control must remove that request from the TUCT (and 
CTUT) and determine the clusters that were locked by that request. Finally, 
whenever a new request is received or an old request has completed execution, 
concurrency control must try to convert as many locks in the clusters re- 
quired by that request to the "being-used" category. When all locks required 
by a request have been converted to "being-used", the process must notify re- 
cord processing to begin execution of the request. 
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5.0 TESTING MDBS 

In order to test MDBS two types of sample information must be made 
available. They are sample databases and sample requests. Therefore, a test 
of MDBS consists of loading a sample database and then executing one or more 
sample requests on the database. 



5.1 The Need for the Generation of Test Databases and Lists of User 

Requests 

In the first report [Kerr82] , it was argued that a program to generate 
test databases would facilitate the testing process. A program, the Test 
File Generation Package . was developed for this purpose. It was also des- 
cribed in the first report. 

A second program, the Test Request Generation and Execution Package . is 
being developed. This program is to assist in the generation of lists of 
sample requests to be executed and to facilitate the execution of the re- 
quests in a test session. In the following sections, we describe this Pack- 
age. 



5.2 The Generation of User Requests Lists 

Several methods of generating lists of requests are possible. In addi- 
tion, once the requests have been generated, several schemes for executing 
the requests are also possible. 



5.2.1 User-Generated vs. Program-Generated Requests 

As with the test files, a user may directly generate each request to be 
executed or a program may generate random requests based on some criteria 
chosen by the user. Since we anticipate our initial tests will use only a 
small number of requests and since we want to choose our requests to test 
certain features of MDBS, we have chosen to implement a package which first 
assists the user in the generation of short lists of requests and then facil- 
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itates the execution of lists of requests intended for certain features of 
MDBS. In other words, we are developing a package for user-generated test 
requests. Program-generated lists of requests are needed for performance 
evaluation experiments but are not needed for testing the features of MDBS. 
Such a package will be developed at a later date. 



5.2.2 A Simple Test Package for a Single User 

The first package developed is intended for testing Version I of MDBS, 
i.e., it assumes a single user wants to execute one request at a time. This 
package first assists the user in the generation of lists of requests. Once 
a list of requests has been generated it is saved in a file so that it can be 
executed at a later time. Thus the user does not have to type in sample re- 
quests repeatedly. 

The test package works as follows. A user decides to have a test 
session consisting of several test subsessions . During each subsession the 
user can do one of the following: 

(1) Execute a list of requests that was previously stored in a file. 

(2) Generate a list of requests to be stored in a file for later use. 

(3) Retrieve a list of requests that were previously stored in a file and 
then select requests from that list for execution. This selection 
can be done in any order. The user will also be able to enter a new 
request to be executed. 

(4) Modify an existing list of requests that was previously stored in a 
file. 

The user can continue with as many subsessions as desired. The user is 
also given a choice of two ways to examine the responses from MDBS. They may 
be displayed immediately at the user terminal and/or they may be saved in a 
file for later examination. The design of this package is given in 
Appendix D. 
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5.2.3 A Test Package for the Simulation of Multiple Concurrent Users 

MDBS is, of course, designed to allow concurrent execution of requests 
by multiple users. Thus all versions of MDBS, except MDBS-I, must be tested 
with multiple concurrent users. In order to perform these tests there must 
be a way to simulate multiple users. 

The simplest technique is to execute multiple copies of the package des- 
cribed in the previous section. Thus, if we had n copies of the package, we 
could simultaneously execute n different lists of requests - one for each 
concurrent user. Although easy to implement, there are two problems with 
this technique. First, setting up the n copies will be inconvenient, since 
we will either need n people sitting at different terminals or someone will 
have to run among a group of terminals. Second, replicating a test will be 
difficult. MDBS merges the requests as they are received from different 
users. The requests are then executed in this merged order, subject to al- 
terations due to concurrency control restrictions. Even if in two tests the 
users all submit the same requests in the same order, there is no guarantee 
that MDBS will receive the combined requests in the same order. Thus the 
merged lists will be different and the two tests will not be identical. 
Although it will be possible to run the same sets of requests, it will be im- 
possible to assure that MDBS will receive the requests in the same order. 
Thus MDBS will not be asked to perform exactly the same sequence of requests. 

An alternative to running multiple copies of a package which can only 
simulate a single user is to run a new package that actually simulates multi- 
ple users. Such a package may be a modification of the single-user , package. 
The main modification would be to associate each request with a particular 
user. Then the requests could be executed in turn thus simulating a 
multi-user system. We plan to use this approach for testing the later ver- 
sions of MDBS. 



5.2.4 A Test Package for the Generation of Random Requests 

Like the Test File Generation Package, which generates test files with 
specified distributions of data values, this package generates certain types 
of requests for performance evaluation. Lists of requests vary in the mix- 
ture of the different request types they contain. Thus a user should be able 
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to specify the percentage of RETRIEVE, INSERT, DELETE and UPDATE requests to 
be generated. In addition, each non- insert request has a query part. Some 
queries may be simple, say with one or two predicates. Others may be more 
complex, say with 10-15 predicates. Thus the user should also be able to 
specify the complexity of the requests being generated. 

MDBS is likely to be more effective handling some forms of requests than 
others. Thus, it is desirable to perform experiments with different distri- 
butions of the request types. A package for the generation of random re- 
quests is to be developed for performance evaluation studies 0 
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6.0 OUR SOFTWARE ENGINEERING EXPERIENCE 

Well-known software engineering techniques have been applied to the de- 
velopment of application programs and the writing of compilers and operating 
systems. They have not, however, been widely applied to database system im- 
plementations. Our goal is not limited to the production of a prototype 
MDBS, but is aimed toward application of software engineering techniques to 
the development of the system. In the application, we are trying to identify 
the adequacy and applicability of the software engineering techniques used. 
We also attempt to modify the existing software engineering methodologies and 
propose new methodologies to tailor them for effective software engineering 
of database systems. In [Kerr82] , we described the initial techniques that 
we were going to use. In this chapter, we describe some of the techniques 
that have been most effective. We also describe the new techniques that we 
have added to our initial techniques. 

We conclude this report by giving the current status of the implementa- 
tion. 



6 .1 The Effectiveness of the Techniques Used 

Different software engineering techniques have been used in the develop- 
ment of MDBS. They include a modified chief-programmer team organization, 
uniform documentation standards, a formal system specification language, use 
of structured walkthroughs, incremental development, top-down design strategy 
combined with the use of data and service abstractions, structured coding and 
a testing approach. Our finding is that most of the techniques may be used 
in prototyping the database system, i.e., MDBS. In this section, we describe 
the techniques that have been most effective. 



6.1.1 The Use of Structured Walkthroughs 

A structured walkthrough is a formal review of the software development 
effort at a given stage in its development cycle. The work is reviewed by a 
walkthrough committee, with the purpose of finding any errors that may be 
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present* The purpose of a walkthrough is not to solve problems, only to 
identify them; neither is a walkthrough a management tool to evaluate any 
employee's performance* 

We have been using this technique at both the design stage and the cod- 
ing stage. All detailed program specifications and source codes are reviewed 
by walkthrough committees. The status of a task can be determined by review- 
ing the walkthrough reports for that task. Figure 16 shows a sample walk- 
through report. A good reference describing the structured walkthrough tech- 
nique is [Your79a] . 

The use of structured walkthroughs has helped us to identify most of the 
design and program code problems. Furthermore, most of the suggestions made 
by the reviewers in the walkthroughs have been very useful to the presenters. 
A presenter, of course, investigates the comments and suggestions made about 
his work, instead of simply modifying his work to incorporate the sugges- 
tions . 

6.1.2 The Use of a Formal Systems Specification Language (SSL) 

Our original design methodology was a systems specification language 
(SSL) modeled on the process design language (PDL) described in [Ling79] . 
The original SSL is described in [Kerr82] . The SSL which we now use is based 
on our original SSL and it is intended to describe systems of any size. The 
current SSL is characterized by a number of constructs for the expression of 
the different levels of a system: system, subsystem, module and procedure. 
A system is at the highest level of the hierarchy. MDBS, for example, is a 
system. 

At the second highest level of the hierarchy, we have the level of sub- 
system. A subsystem is a separate component of a system. In other words, 
each system may consist of several subsystems. The MDBS controller, for ex- 
ample, is a subsystem, as is each MDBS backend. The system, consisting of 
the controller and the backends, is the MDBS. 



Below the level of subsystem, we have the level of module. A module is 
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intended for the implementation of a data or service abstraction. It con- 
sists of the procedures and data structures implementing the abstraction. A 
procedure is at the lowest level of the hierarchy. It corresponds to the 
usual notion of a subroutine. Procedures are invoked to perform some work on 
some input data and produce some output. However, they are not allowed to 
retain data between invocations. A formal outer syntax and an informal inner 
syntax are used in a procedure. The outer-syntax allows only the following 
three types of constructs: sequence, decision and iteration. Below is an 

example of the if-then-else decision construct. 
if expression 

then statement sequence 
else statement sequence 

endif 

The underlined words represent the formal outer syntax. The other words 
represent the informal inner syntax; the only requirement for this inner 
syntax is that it must be understood by all project members. Figure 17 shows 
a typical SSL procedure specification. 

The use of a formal system specification language has been very effec- 
tive. More specifically, by using the SSL: 

(1) Precise and unambiguous communication among the project members is 
achieved . 

(2) Complete and accurate documentations are produced. 

(3) Dependence on individuals is reduced. 

(4) Project management is easier. 

One useful concept that we have not employed is multi-level data ab- 
stractions (having higher level data abstractions which use lower level data 
abstractions which in turn use lower level data abstractions, and so on). We 
have used data abstractions only at the lower levels. The reason for this is 
probably that we are not used to the concept of multi-level data abstrac- 
tions. This concept, however, leads to better and well-structured design. 
Thus, it should be employed. 
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FOURTH LEVEL SPECIFICATION FOR DATA3ASE LOAD 
VERSION 2) September 1 6? 1931 




\ 



4*10*21*1 



2 »ac LIST.TYPE-C-ATTR.NAMES /X TYPECLST (DBLllio ) X/ 

(input: type-C-attr-.oaces? 
atpointer ) ) 

JX List all the attribute nanes over which type-C descriptors 
/x are to be defined. Input, is a list for attribute names 
/X over which type-C attributes are do be^ defined? and a 
/X pointer to the AT. 




4*10.21.2 



scalar 



index? /X Index to list of attribute 

T— H3 Tl Q f 

duplicate? /% Indicator. - TRUE or FALSE, 
dditpointer?/'*: Pointer into DOIT returned 
/x FIND function. 

descr_type ? /x A? 3? C? or NQTFOUND* . 



names < 



X/ 



XJ 

from ATMs/ 
x/ 

x/ 



4.10.21*3 

4.10*21.4 

■4.10*21.5 

4.10.21.6 

4.10.21 .7 
4.10.21.3 



4,10.21.? 



4.10.21.10 

4.10.21.11 

4.10.21.12 

4.10.21.13 
4, 10,21.1 4*2 
4.10.21.15 



4.10.21.13' 

4.10.21.17 

4.10.21.13 

4.10.21.1? 

4.10.21.20 

4.10.21.21 

4.10.21.22 

4.10.21.23 

4.10.21.24 



4.10.21.25 sod 



index 



/X Null indicates end of list. 



tsP9-C_3ttr_nanesCinde:<] := null? 

while-,. nore type-C descriptors da 
■ 2 • i 2 — 




Set attr.name from terminal? 
eetiaro ATHSF1ND ( attr_name ? 

• dditpointer? 
pointer to descr.type) i 

Li a type-A or type-B descriptor is already defined 
over this attribute name 
/X descr_type not = NOTrGUND x/ 

thee 

display error messag e ?‘ f _r.ner syntax e_.ements- 

else _ — — ■ — " \ are not underlined 

beeio 

duplicate = -AL3E5 
eeciaca SEARCH- TYPE-C_ATTR_N AMES 
( typ9-C_3ttr_natties ? 
att r_name ? program constant 

dupl icate ) J 

Li duplicate is FALSE' 
idea 

beeio 

type-C_ 3 tt r_naaesCindex3 l- attr_nsae? 
index 1= index ? 1 ?' 
type-C_3t tr_nanesCindex] ! = null? 
acd-ii? 
eod-ii? 





procedure number is 4.10.21 which means that it < 
was called at orogram statement 21 in the level-3 j. 
orocedure numbered 4.10. That procedure was in turn, cal.' 
at orogram statement 10 of the level-2 orocedure numbered 
Procedure 4, in turn, was called by program statement 
main orocedure. 



Figure 17. A SSL Specification of a Program Procedure 
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6.1.3 A Top-Down Design Strategy and the Use of Data and Service 
Abstractions 

A top-down design strategy is a natural choice for MDBS. The design and 
analysis study in [Hsia81a] and [Hsia81b] clearly describes the top level of 
design. It also suggests the possibility of functional decomposition, i.e., 
the entire system can be broken into discrete functional units. For example, 
the execution phases of a retrieval request can be broken down into directory 
management and record processing, as depicted in Figure 2. Directory manage- 
ment, an example of a functional unit, includes the descriptor search, clus- 
ter search, and address generation phases of request execution. 

At a lower level, one concept, data and service abstractions, is used 
which originated in the bottom-up design approach. Since MDBS is being de- 
veloped as a prototype system and is aimed for research into performance eva- 
luation, we anticipate that data structures and system services will be rou- 
tinely modified in attempts to measure the effect of different data struc- 
tures on system performance* The abstractions allow us to separate the basic 
system functions from the data structures and from the implementation of the 
services, minimizing the effect on the system when data structures or imple- 
mentation services are modified. 

We did not, unintentionally, follow the top-down design strategy in the 
development of the MDBS controller. Instead, the functional components of 
the controller such as Request Composer and Reply Monitor were first deter- 
mined. Then, the categories of functions such as Request Preparation and 
Post Processing were determined based on the functional components. A 
top-down design of the controller would have first identified the categories 
of functions such as Request Preparation, Insert Information Generation and 
Post Processing. It would have then decomposed them into smaller components. 



6.2 Trying New Software Engineering Techniques 

We have added new software engineering techniques to our initial techni- 
ques. In this section, we describe these new techniques. 
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6.2.1 The Use of Jackson Charts 

Our original designs were developed using only SSL. More recently, we 
have begun using a technique, Jackson chart [ Jack75] , to represent the pro- 
gram structures. Three constructs are used in a chart: 

(1) Sequence - Figure 18a shows a sample sequence. In this example, the 
sequence A consists of B followed by C followed by D. 

(2) Iteration - Figure 18b shows a sample iteration. In this example, 
the iteration A consists of multiple occurrences of B. 

(3) Selection - Figure 18c shows a sample selection. In this example, 
the selection A consists of one of B, C or D. 

A sample program structure and its corresponding SSL are shown in Figures 19 
and 20, respectively. 

Jackson charts contain fewer details than the SSL specifications, and 
provide a two-dimensional representation of program structure. These charts, 
along with the SSL specification, are used to document the detailed design. 



6.2.2 Standards for Module Decomposition 

As explained in the previous chapters, the entire MDBS system has been 
designed as a set of discrete functional units. We propose to apply the same 
idea of functional decomposition at the level of subsystem design. We need 
some way to evaluate the modularity of our decomposition. This need became 
apparent when we began designing the top-level scheme for MDBS subsystems. 
It is necessary that we develop a unified view of the overall function of the 
subsystems of the controller and the backends before proceeding to design the 
abstractions and procedures. We have added to our collection of software en- 
gineering strategies two measures of modularity, or functional decomposition. 

The first of these measures is strength of module cohesion [Your79b]. 
Cohesion is defined as the relatedness of processing elements within a single 
component of a system, i.e., a subsystem, a module or a procedure. The de- 
gree of relatedness determines the level of cohesion . Several levels of 
cohesion, ranked from least to most desirable, are recognized. A component 
is said to be functionally cohesive , most desirable level of cohesion, if 
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Figure 19. A Sample Program Structure 
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10.1 proc DELETE ?RQCZSSING( input : QUERY, ADDRESSES ) ; 

/* This procedure is to be used for processing of DELETE requests. */ 

10.2 list QUERY: string; 

10.3 set ADDRESSES: integer; 

10.4 array TRACX_3UFFER : word; 

10.5 scalar indexA, indexB: integer; 

/*These are pointers for ADDRESSES and TRACX_ 3UFFER respectively */ 

10.6 scalar satisfied: logical; 

/* Process data track by track . */ 

10.7 for each address ADDRESSES ( indexA) in ADDRESSES do 

/* Fetch one track into TRACX_3UFFER . */ 

10 . S perform FETCH_T0_TRACX_3UFFER ( indexA , TRACX_3UFFER ) ; 

/* Select records in TRACX_3UFFER one by one. */ 

10.9 for each record TRACX_3UFFER( index3) in TRACX_BUFFER do. 

10.10 if. the record is not marked for deletion 

10.11 then begin 

/* Check whether the record satisfies the QUERY. */ 

10.12 oerrorm CEECX_ QUERY ( QUERY , TRACX_3UFFZR , indexB, satisfied); 

10.13 if. satisf ied=" true" 

then 

/* Mark the retrieved record in TRACX_ 3UFFER( indexB) .*/ 

10 .14 perform DELETE (TRACX_3UFFER , indexB) ; 

10.15 end if 

10.16 end begin 

10 .17 end if 

10.13 end for /* indexB */ 

/* Store TRACX_3UFFER back to disic. */ 

10.19 perform ST0RE_TRACX_3UFFER( indexA, TRACX_3UFFER ) ; 

10.20 end for /* indexA */ 

10 .21 end proc 



Figure 20. The SSL Corresponding to the Sample Program Structure in Figure 3 
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"every element of processing is an integral part of, and is essential to, the 
performance of a single function". An ad hoc measurement is that the des- 
cription for a functionally cohesive component should consist of one impera- 
tive sentence containing one transitive verb and one non-plural object. We 
have applied this ad hoc measurement to our current design work. The de- 
signer is required to write a functional description of each program compo- 
nent, say an abstraction, of the design. Each description begins with a sin- 
gle imperative sentence which concisely describes the function of that compo- 
nent. For example, the following sentence describes the function of a proce- 
dure of the MDBS controller, the Aggregate Post Operation 

Aggregate Post Operation performs the final aggregation 
operation on tne partial aggregate results returned 
from the backends. 

The second measure of modularity is the degree of interconnections 
between components. Coupling is a measure of the strength of interconnection 
between components. Several categories of coupling, ranked from lowest (best 
case) to tightest (worse case), are recognized. Two components are said to 
have no. direct coupling . lowest coupling (best case) , if each can function 
without knowledge of the other. We now give an example to show how we em- 
ployed these standards for module decomposition. The original design for the 
controller, Figure 21, had a function called Insert/Update Information Gener- 
ator. This function was intended to perform the following operations: 

(1) to select a backend for record insertion when executing an insert re- 
quest 

(2) to generate new descriptor ids 

(3) to generate new cluster ids. 

In addition to performing the above operations by itself, Insert/Update In- 
formation Generator was intended to perform the following operations together 
with Request Composer: 

(1) to initiate the actions required for the insertion of the records 
that change cluster as a result of executing an update request 

(2) to generate update requests with type-0 modifier for update requests 
with type-III or type-IV modifier. 

This function. Insert /Update Information Generator, is not functionally cohe- 
sive and it is highly coupled with Request Composer. Thus, we changed the 
original design of the controller. 
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6.3 Current Status of the Implementation 

The implementation of MDBS is underway. The detailed design of the con- 
troller is finished and coding has begun. The parser has been coded and 
tested. The first version of directory management is complete. The design 
for the second version has begun. The detailed design of record processing 
is finished and coding has begun. The design of concurrency control has 
begun. In addition to MDBS itself, we have completed several required auxi- 
liary programs such as a database load utility. We expect to begin testing 
MDBS-I soon. We expect to finish MDBS-II, III and IV by the end of this 
year. 
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In Appendices B, C and D, a large number of MDBS programs are described 
and specified. These programs represent those parts of MDBS that have been 
designed since the first report in this series was written. 



A.l Parts within an Appendix 

Each appendix begins with an introduction which outlines the major com- 
ponents of the design. For example, the design of the test request genera- 
tion and execution package, presented in Appendix D, consists of two major 
components: one to generate lists of requests to be executed and then to ex- 
ecute those requests, the other to handle the output resulting from the exe- 
cution of the requests. Accordinngly , each major component is described and 
specified in a separate part of the appendix. Thus Appendix D has Part I and 
Part II. 



A. 2 The Format of a. Part 

In each part, we provide the following documentation elements: 

(1) Title of the part, 

(2) Name of the design, 

(3) Name of the designer, 

(4) Date the design was first submitted, 

(5) Dates of design modifications, 

(6) Statements of the design purpose, and of the input and output re- 
quirements , 



PAGE 83 



(7) Formal specifications of the input and output, if necessary, 

(8) Procedure names used in the design, 

(9) Jackson chart of the design, 

(10) Data structures used in the design, 

(11) Program specification of the design. 



A. 3 Documentation Techniques for a. Part 

In the previous section, we listed the various documentation elements. 
They are used to describe a design. Documentation elements 1 through 5 are 
written in English phrases. Document element 6 is written in prose. On the 
other hand, document elements 7 through 11 can be expressed more effectively 
using other means. Specifically, we use Backus-Naur form (BNF) for writing 
the specifications in document element 7. 

The procedure names of document element 8 are shown in a program hierar- 
chy. The use of the hierarchy makes clear the calling sequences of the pro- 
cedures named. A Jackson chart as described in Section 6.2.1 and depicted in 
Figure 19 appears as element 9. The data structures of documentation element 
10 are specified in either SSL or in the C programming language. In documen- 
tation element 11, the procedures, themselves, are specified in SSL. 

Except for the programming team that writes the procedures, other teams 
will usually not be interested in the internal logic of the procedures. 
Consequently, they need only know the higher-level specifications of the pro- 
cedures. SSL as described in Section 6.1.2 and depicted in Figure 17 is an 
ideal specification language for revealing the design of the procedures from 
a top-to-bottom-and-layer-to-layer way. It also works well with the hier- 
archical organization of procedures. 
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APPENDIX B 

THE SSL SPECIFICATION FOR MDBS CONTROLLER 



The system specification for the controller is given in this appendix. 
The specification consists of three parts, one for each process (task) in the 
controller . 

In Part I, the Request Preparation process is specified. Insert Infor- 
mation Generation process and Post Processing process are specified in Parts 
II and III, respectively. 



B.l 


Part I - The Request Preparation Process 




/* 


(1) Part I 


: The Request Preparation Process 


*/ 


/* 


(2) Design 


: REQUEST-PREPARATION 


*/ 


/* 


(3) Designers 


: A. Orooji, Z.Z. Shi, P .R. .Strawser 


*/ 


/* 


(4) Date 


: Feb. 4, 1982 


*/ 


/* 


(5) Modified 


: Feb. 19, 1982 


*/ 


/* 




Apr. 12, 1982 


*/ 


/* 




Apr. 26, 1982 


*/ 


/* 




May 13, 1982 


*/ 


/* 




May 19, 1982 


*/ 


/* 


(6) Purpose 


: 


*/ 


/* 


This is the 


Request Preparation process. It consists of 


*/ 


/* 


the functions which must be performed before a request or a 


*/ 


/* 


transaction can 


be broadcasted to the backends. 


*/ 
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(8) Procedure Hierarchy for REQUEST-PREPARATION 
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(9) Program Structure of REQUEST-PREPARATION (Jackson Chart) 
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(10) Data Structures 

/* The data structure definitions are included in the program */ 
/* specifications. */ 



(11) Program Specifications 



1 . task REQUEST-PREPARATION 

/* TrafficUnit : */ 
/* is either a request or a transaction */ 
/* traffic-id : */ 
/* an integer that identifies a traffic unit */ 
/* Traff icUnitPtr : */ 
/* a pointer to (traff ic-id , Traff icUnit) */ 
/* AllCorrect : */ 
/* indicates whether or not all the requests in the traffic */ 
/* unit are syntactically correct */ 
/* request-id : */ 
/* consists of (traff ic-id, request-no) which uniquely */ 
/* identifies each request being processed by MDBS */ 
/* ParsedRequestsPtr : */ 
/* is one of the following two */ 
/* (1) a pointer to .( (request-id .routing- indicat or , */ 
/* no-pred, parsed-request) [,...] ) if all the requests*/ 
/* in the traffic unit are syntactically correct. */ 
/* (2) a pointer to ( (request-id .request , */ 
/* error-message) [,...] ) if one (or more) of the */ 
/* requests in the traffic unit is not syntactically */ 
/* correct. */ 
/* FormattedRequestsPtr : */ 
/* a pointer to ( (request-id .routing-indicator , no-pred , */ 
/* sched-no .formatted-request) [,...] ) */ 
/* AggregateOperatorsMessagePtr : */ 
/* a pointer to ((request-id, (attribute, */ 
/* aggregate-operator) [,...]) I ,...] ) */ 
/* RequestCountMessagePtr : */ 
/* a pointer to (traff ic-id, request-count) */ 
/* request-count : */ 
/* number of requests in a traffic nuit */ 
/* MessageType : */ 
/* indicates the type of a message */ 
/* MessagePtr : */ 
/* a pointer to a message */ 



2 . 

3. 

4. 



do initialization work; 

while 'true' do. /* do forever */ 

if according to the task scheduling this task should 

release the processor 



then 

release the processor and wait; 

end if 

/* get the next message for REQUEST-PREPARATION */ 
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8 . 



9. 

10 . 
11 . 

12 . 

13. 

14. 

15. 



16. 

17. 

18. 
19. 



20 . 

21 . 



22 . 



23. 

24. 



25. 



26. 



27. 

28. 



29. 



perform RP-RECEIVE$MESSAGE ; 

/* get the sender name of the next message for REQUEST- */ 

/* PREPARATION */ 

perform RP-RECEIVE$SENDER( sender ) ; 
case sender value 
'host machine': 

/* get the traffic unit */ 

perform RP-RECEI VE $TRAFFIC-UNIT ( Traf f icUni tP tr ) ; 

/* parse all the requests in the traffic unit */ 

perform PARSER(Traff icUnitPtr ,AllCorrect , ParsedRequestsPtr , 

AggregateOperatorsMessagePtr ,RequestCountMessagePtr ) ; 
if AllCorrect 
then 

/* all the requests in the traffic unit are */ 

/* syntactically correct */ 

/* send number of requests in the traffic */ 

/* unit to POST-PROCESSING ' */ 



MessageType : = 'number-of-requests-in-a- 

traffic-unit message'; 
perform SEND-POST-PROCESSING(MessageType , 

RequestCountMessagePtr) ; 
if there are aggregate operators 
then 

/* send the aggregate operators in the */ 

/* requests to POST-PROCESSING */ 

MessageType := 'aggregate-operators message'; 
perform SEND-POST-PROCESSING(MessageType , 

AggregateOperatorsMessagePtr) ; 

end if 

/* transform the requests into the form required */ 

/* for processing at the backends */ 

MessageType := 'PARSER message'; 
perform REQUEST-COMPOSER(MessageType , 

ParsedRequestsPtr ,FormattedRequestsPtr ) ; 

/* send the requests to all the backends */ 
perform BROADCAST-ALL-DIRECTORY -MANAGEMENT ( 

FormattedRequestsPtr) ; 



else 

/* one (or more) of the requests in the traffic */ 
/* unit has errors */ 
/* send the requests along with error messages */ 
/* to POST-PROCESSING */ 



MessageType := 'request-with-error message'; 
perform SEND-POST-PROCESSING(MessageType , 

ParsedRequestsPtr) ; 



end if 



30. 'a backend': 

/* There is a message from a backend for REQUEST- */ 
/* PREPARATION when */ 
/* - a retrieve request caused by an update */ 
/* request is completed */ 



/* - a record has changed cluster when executing */ 
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/* an update request. */ 

/* get the message sent by the backend */ 

31. perform RP-RECEIVE$BE-MESSAGE(MessagePtr) ; 

/* Generate a request. (This request is either an */ 

/* insert or an update) */ 

32. MessageType := 'backend message'; 

33. perform REQUEST-COMPOSER(MessageTvpe . 

MessagePtr , FormattedRequestsPtr) ; 
/* send the request to all the backends */ 

34. perform BROADCAST-ALL-DIRECTORY-MANAGEMENT ( 

FormattedRequestsPtr) ; 

35. 'otherwise': 

36. system error; 

37 . end case 

38. end while 



13.1 proc PARSER( input : Traf f icOnitPtr , 

output : AllCorrect ,ParsedRequestsPtr , 

AggregateOperatorsMessagePtr ,RequestCountMessagePtr) ; 
/* This routine parses all the requests in a traffic unit. (A */ 



/* traffic unit is either a request or a transaction.) */ 

/* */ 
/* Traff icUnitPtr : • */ 

/* a pointer to (traff ic-id, Traff icUnit) */ 

/* AllCorrect : */ 

/* indicates whether or not all the requests in the traffic*/ 

/* unit are syntactically correct */ 

/* ParsedRequestsPtr : */ 

/* is one of the following two */ 

/* (1) a pointer to ((request-id, routing-indicator , */ 

/* no-pred ,parsed-request) [,...] ) if all the */ 

/* requests in the traffic unit are syntactically */ 

/* correct. */ 

/* (2) a pointer to ( (request-id .request , */ 

/* error-message) [,...] ) if one (or more) of the */ 

/* requests in the traffic unit is not syntactically*/ 

/* correct. */ 

/* AggregateOperatorsMessagePtr : */ 

/* a pointer to ( (request-id , (attribute , */ 

/* aggregate-operator) [,...]) I ,...] ) */ 

/* RequestCountMessagePtr : */ 

/* a pointer to (traff ic-id, request-count) */ 

/* request-count : */ 

/* number of requests in a traffic nuit */ 

13 .2 end proc 



24.1 proc REQUEST-COMPOSER ( input : MessageType , MessagePtr , 

output : FormattedRequestsPtr); 

/* This routine transforms the requests into the form required*/ 
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/* for processing at the backends. */ 

/* */ 
/* MessageType : */ 

/* indicates the type of a message */ 

/* MessagePtr : */ 

/* a pointer to a message. (It is either ParsedRequestsPtr */ 

/* or a pointer to a message sent by a backend.) */ 

/* ParsedRequestsPtr : */ 

/* a pointer to ( (request-id , routing-indicator ,no-pred , */ 

/* parsed-request) [,...] ) */ 

/* FormattedRequestsPtr : */ 

/* a pointer to ( (request-id , routing-indicator , */ 

/* no-pred,sched-no,formatted-request) [,...]) */ 

24.2 end proc 



module RP-RECEIVE 

programs MESSAGE, SENDER, TRAFFIC-UNIT, BE-MESSAGE 
datasets MessageBuf f er 

/* used to store messages for REQUEST-PREPARATION */ 

8.1 £roc MESSAGE ( input : nothing, output : nothing ); 

/* This routine gets the next message for REQUEST-PREPARATION */ 
/* and stores it in MessageBuf fer . */ 

8.2 end proc 



9.1 proc SENDER( input : nothing, output : sender); 

/* This routine returns the sender name of the next message */ 
/* for REQUEST-PREPARATION */ 

/* */ 

/* sender : */ 

/* the sender name of the next message for REQUEST- */ 

/* PREPARATION */ 

9 .2 end proc 

12.1 proc TRAFFIC-UNIT( input : nothing, output : Traf f icUnitPtr ) ; 

/* This routine returns a pointer to the next traffic unit for*/ 
/* REQUEST-PREPARATION */ 

/* */ 

/* Traff icUnitPtr : */ 

/* a pointer to ( traf fic-id , Traf ficUnit) */ 

12.2 end proc 

31.1 proc BE-MESSAGE( input : nothing, output : MessagePtr); 

/* This routine returnes a pointer to the next message for */ 

/* REQUEST-PREPARATION sent by a backend. There is a message */ 

/* from a backend for REQUEST-PREPARATION when */ 

/* - a retrieve request caused by an update request is */ 

/* completed */ 

/* - a record has changed cluster when executing an update */ 

/* request */ 

/* */ 

/* MessagePtr : */ 
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/* a pointer to a message */ 

31 .2 end proc 

end module 

17.1 proc SEND-POST-PROCESSING( input : MessageType .MessagePtr , 

output : nothing); 

/* This routine sends a message to POST-PROCESSING */ 

/* */ 

/* MessageType : */ 

/* indicates the type of a message */ 

/* MessagePtr : */ 

/* a pointer to a message */ 

17 .2 end proc 

25.1 proc BROADCAST-ALL-DIRECTORY-MANAGEMENT ( input ; FormattedRequestsPtr , 

output : nothing); 

/* This routine broadcasts a set of formatted requests to all */ 

/* the backends */ 

/* FormattedRequestsPtr : */ 

/* a pointer to ((request-id, routing-indicator ,no-pred, */ 

/* sched-no ,formatted-request) [,...] ) */ 

25.2 end proc 

39. end task 



B.2 Part II - The Insert Information Generation Process 

/* (1) Part II : The Insert Information Generation Process */ 
/* (2) Design : INSERT-INFORMATION-GENERATION */ 
/* (3) Designers : A. Orooji, Z.2. Shi, P.R. Strawser */ 
/* (4) Date : Feb. 4, 1982 */ 
/* (5) Modified : Feb. 19, 1982 */ 
/* Apr. 12, 1982 */ 
/* Apr. 26, 1982 */ 
/* May 25, 1982 */ 
/* May 28, 1982 */ 
/* Jun. 2, 1982 */ 
/* (6) Purpose : */ 
/* This is the Insert Information Generation Process. It */ 
/* consists of the functions which must be performed during the */ 
/* processing of an insert request to furnish additional */ 
/* information required by the backends. */ 
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(8) Procedure Hierarchy for INSERT- INFORMATION-GENERATION 
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(9) Program Structure of INSERT-INFORMATION-GENERATION (Jackson Chart) 
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Program Structure of BACKEND-SELECTOR (Jackson Chart) 
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(10) Data Structures 

/* The data structure definitions are included in the program */ 
/* specifications. */ 



(11) Program Specifications 



1 . task INSERT-INFORMATION-GENERATION 

/* request-id : */ 
/* consists of (traff ic-id , request-no) which uniquely */ 
/* identifies each request being processed by MDBS */ 
/* routing-indicator : */ 
/* indicates where results /messages from backends should go to*/ 
/* last : */ 
/* a flag ('true' or 'false') used when processing an insert */ 
/* request. This value is returned by BACKEND-SELECTOR and it */ 
/* indicates whether or not all the backends have returned a */ 
/* cluster id (or a null value). */ 
/* exist : */ 
/* a flag ('true' or 'false') used when processing a request */ 
/* for a new descriptor id. This value indicates whether or */ 
/* not the descriptor already exists. */ 
/* NewTrack : */ 
/* a flag ('true' or 'false') used when inserting a record. */ 
/* This value indicates whether or not this is the first */ 
/* record being inserted in a track. . */ 



do initialization work; 
while 'true' do_ /* do forever */ 

if according to the task scheduling this task should 

release the processor 

5. then 

6. release the processor and wait; 

7 . end if 

/* get the next message for INSERT-INFORMATION-GENERATION */ 

8. perform IIG- RECEIVE $MESSAGE ; 

/* get the routing-indicator in the next message */ 

/* for INSERT-INFORMATION-GENERATION */ 

9. perform IIG-RECEIVE $ROUTING-INDICATOR( rout ing-indica tor ) ; 

10. case routing-indicator value 

11. 'BACKEND-SELECTOR': 

/* an insert request is being executed */ 

/* receive the request-id and the cluster id (or a */ 
/* null value) returned by a backend */ 

12. perf 3rm IIG-RECEIVE$CLUSTER-ID(request-id , cluster-id) ; 

/* Determine a backend for record insertion if all */ 
/* the backends have returned a cluster id (or a */ 
/* null value). Otherwise, save the cluster id (or */ 
/* null value) returned by the backend. */ 

/* (BACKEND-SELECTOR will call CLUSTER- ID-GENERATOR*/ 
/* when there is a new cluster.) */ 

perform BACKEND-SELECTOR(request-id , cluster-id. 



13 
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14, 

15. 



16. 

17. 



last, backend -no , NewTrack); 



if last 
then 

/* All the backends have returned a cluster */ 

/* id (or a null value). */ 

/* Send the backend number selected for */ 

/* record insertion to all the backends */ 

perform BROADCAST-ALL-ADDRES S-GENERATION( request- id , 
backend-no, cluster-id, NewTrack); 



end if 



18. 



19. 



20 . 

21 . 

22 . 

23. 

24. 



'DESCRIPTOR- ID -GENERATOR' : 

/* request for new type-C subdescriptor */ 

/* receive the request-id and the descriptor */ 
perform IIG-RECEIVE$DESCRIPTOR( request-id , descriptor ) ; 

/* The request for new type-C subdescriptor might */ 

/* have already been received from another backend.*/ 

/* In that case, the descriptor id has already been */ 

/* generated and broadcasted. */ 

/* Generate a descriptor id if it is not already */ 

/* generated */ 

perform DESCRIPTOR-ID-GENERATOR(descriptor , 

exist , descriptor-id) ; 

if exist - 'false' 
then 

/* broadcast the descriptor id to all the backends */ 
. perform BROADCAST-ALL-DESCRIPTOR-SEARCH( request-id , 

descriptor , descriptor- id) ; 

end if 



25. 'otherwise': 

26. system error; 

27 . end case 

28. end while 



13.1 proc BACKEND-SELECTOR( input : request-id, input /output : cluster-id, 

output : last , backend -no , NewTrack ); 

/* This routine is called whenever a backend returns a cluster */ 



/* id (or a null value). It determines a backend for record */ 
/* insertion when all the backends have returned a cluster id */ 
/* (or a null value). Otherwise, it saves the cluster id (or */ 
/* null value) returned by a backend. */ 
/* (This routine will call CLUSTER- ID-GENERATOR when there is */ 
/* a new cluster.) */ 
/* */ 
/* request-id : */ 
/* consists of ( traffic-id, request-no) which uniquely */ 
/* identifies each request being processed by MDBS */ 
/* cluster- id : */ 
/* uniquely identifies each cluster */ 
/* NoBackends : */ 
/* total number of backends in MDBS. (This is a variable */ 
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13.2 



13.3 

13.4 



13.5 



13.6 

13.7 



13.8 

13.9 

13.10 

13.11 



13.12 

13.13 



/* defined in SYSGEN.) */ 
/* last : */ 
/* a flag ('true' or 'false'). This value indicates whether */ 
/* or not all the backends have returned a cluster id (or a */ 
/* null value). */ 
/* backend-no : */ 
/* the number of the backend minicomputer selected to insert*/ 
/* a new record into the database store */ 



/* NewTrack 



*/ 



/* a flag ('true' or 'false'). This value indicates whether */ 

/* or not this is the first record being inserted in a track*/ 



/* 

* struct request-cluster-id { 

* rid /* (traffic-id, request-no) */ 

* cids-received-count /* an integer indicating number 

* of cluster ids received for request rid */ 

* cid /* cluster id received for request rid */ 

* /* each cluster id received for a request is */ 

* /* either null or has the same value as the */ 

* /* other non-null cluster ids received for */ 

* /* the request */ 

* > 

*/ 

list all-req-clus-id; /* every element of this list consists 

of the three parts in request-cluster-id */ 



if request-id = rid of one of the elements in all-req-clus-id list 
then 

/* this is not the first cluster-id received for request */ 

/* request-id */ 

ptr := pointer to the element of all-req-clus-id 

with rid = request-id; 

/* save the cluster id received from the backend if needed */ 
perform REQ-CLUS-CHECK-SAVE( cluster-id , all-req-clus-id, ptr); 
else 



/* this is the first cluster-id received for request 
/* request-id 

/* add an element to all-req-clus-id list 
perform REQ-CLU S -ELEMENT-ADD ( request-id , 

cluster-id, all-req-clus-id, ptr); 



end if 

/* check to see whether or not all the backends have */ 
/* returned a cluster id (or a null value) */ 

if cids-received-count < NoBackends 
then 

/* not all the backends have returned a cluster */ 
/* id (or a null value) */ 

last := 'false'; 
else 



/* all the backends have returned a cluster id (or */ 
/* a null value) */ 



*/ 

*/ 

*/ 
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13.14 last :« 'true'; 

13.15 ,if cid(ptr) * null 

13.16 then 

/* there is a new cluster */ 

13.17 perform CLUSTER- ID-GENERATOR( cluster- id ) ; 

/* add an entry to cluster-id-to-next-backend table */ 

13.18 perform CINBT-ENTRY-ADD( cluster-id) ; 

13 .19 end if 

/* select a backend for record insertion */ 

13.20 perform CINBT-SELECT(cluster-id , backend-no, NewTrack) ; 

13.21 delete element(ptr) in all-req-clus-id list; 

13.22 end if 

13 .23 end proc 

13.8.1 proc REQ-CLUS-ELEMENT-ADD( input : request-id , cluster-id , 

input /output : all-req-clus-id, output : ptr ); 

/* This routine adds an element to all-req-clus-id list */ 

13.8.2 list all-req-clus-id; 

13.8.3 add an element to all-req-clus-id list; 

13.8.4 ptr :* pointer to the element just added; 

13.8.5 -rid(ptr) :* request-id; 

13.8.6 cid(ptr) cluster-id; 

13.8.7 cids-received-count(ptr) := 1; 

13 .8.8 end proc 



13.6.1 



13.6.2 

13.6.3 

13.6.4 

13.6.5 

13.6.6 

13.6.7 



13.6.8 

13.6.9 



13.6.10 

13.6.11 



13.6.12 

13.6.13 

13.6.14 

13.6.15 

13.6.16 



proc REQ-CLUS-CHECK-SAVE( input : cluster-id , ptr , 

input /output : all-req-clus-id, ouput : nothing); 
/* This routine saves the cluster id received from a */ 

/* backend if needed */ 

list all-req-clus-id; 

cids-received-count (ptr) :* cids-received-count (ptr) + 1; 
if cluster- id null 
then 

/* cluster id received from the backend is not null */ 
if cid(ptr) - null 
then 

/* all the previous cluster ids received from */ 

/* backends are null */ 

/* cluster-id null, cid(ptr) = null */ 
cid(ptr) := cluster-id; 
else 

/* cluster-id “*= null, cid(ptr) null */ 

if cluster-id ~= cid(ptr) 
then 

/* cluster-id null, cid(ptr) null, */ 

/* cluster-id cid(ptr) */ 

'system error'; 

end if 

end if 

end if 
end proc 
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13.18.1 proc CINBT-ENTRY-ADD( input : cluster-id, output : nothing ); 

/* This routine adds an entry to cluster-id-to-next-backend */ 
/* table */ 

13.18.2 add an entry to cluster-id-to-next-backend table; 

/* cluster id for this entry is cluster-id */ 

/* backend number for this entry is a random */ 

/* number bn ( 1 =< bn =< NoBackends ) */ 

13.18.3 end proc 



13.20.1 proc CINBT-SELECT( input : cluster-id, output : backend -no ,NewTrack) ; 

/* This routine selects a backend for record insertion */ 

13.20.2 backend-no :■ backend selected (using CINBT) for record insertion; 

13.20.3 set NewTrack (using CINBT); 

13.20.4 update CINBT if needed; 

13.20.5 end proc 



13.17.1 proc CLUSTER-ID-GENERATOR( input : nothing, output : cluster- id ); 



/* This routine generates a new cluster id. */ 

/* ' */ 
/* cluster-id : */ 

/* uniquely identifies each cluster */ 

13.17.2 end proc 



20.1 



proc DESCRIPTOR- ID-GENERATOR( input : descriptor, 

output : exist , descriptor-id ); 

/* This routine generates a new descriptor id for a descriptor 
/* if it is not already generated. 

/* 

/* exist : 

/* a flag ('true' or 'false'). This value indicates whether 
/* or not the descriptor already exists. 

/* descriptor-id : 

/* uniquely identifies each descriptor 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



/* This first implementation of DESCRIPTOR-ID-GENERATOR keeps */ 
/* the descriptor ids generated from system start up to system*/ 
/* shut down (or until database reorganization). We need to */ 
/* keep the descriptor ids generated because multiple backends*/ 
/* may request a descriptor id for the same descriptor. */ 



/* 



* struct descriptor-descriptor- id { 

* desc /* descriptor */ 

* desc-id /* corresponding descriptor id */ 

* } 

*/ 



list desc-desc-id-list ; /* every element of this list 



20.2 



PAGE 100 



20.3 

20.4 

20.5 

20.6 

20.7 

20.8 

20.9 

20.10 
20.11 



has the two 
if descriptor = desc of 
then 

/* the descriptor 
exist := 'true'; 
else 

/* the descriptor 



elements in descriptor-descriptor-id */ 
one of the elements in desc-desc-id-list 

id is already generated */ 
id is not generated yet */ 



exist := 'false'; 

descriptor- id := generate a descriptor id; 

add (descriptor , descriptor-id) to desc-desc-id-list ; 

end if 
end proc 



module I IG- RECEIVE 

programs MESSAGE, ROUTING-INDICATOR, CLUSTER-ID, DESCRIPTOR 
datasets MessageBuf f er 

/* used to save messages for INSERT-INFORMATION-GENERATION */ 

8.1 proc MESSAGE ( input : nothing, output : nothing ); 

/* This routine gets the next message for INSERT- INFORMATION- */ 

/* GENERATION and stores it in MessageBuf fer . */ 

8.2 end proc 



9.1 proc ROUTING- INDICATOR( input : nothing, 

output : routing-indicator) ; 

/* This routine returns the routing-indicator in the next */ 

/* message for INSERT- INFORMATION-GENERATION. */ 

/* */ 

/* routing-indicator : */ 



/* indicates where results /messages from backends should go to*/ 
9.2 end proc 

12.1 proc CLUSTER-ID( input : nothing, output : request-id, cluster-id) ; 

/* This routine returns the request-id and the cluster id (or a */ 



/* null value) returned by a backend. */ 

/* */ 

/* request-id : */ 

/* consists of (traffic-id, request-no) which uniquely */ 

/* identifies each request being processed by MDBS */ 

/* cluster-id : */ 

/* uniquely identifies each cluster */ 

12.2 end proc 

19.1 proc DESCRIPTOR( input : nothing, output ; request-id , descriptor ) ; 

/* This routine returns the descriptor sent by a backend. */ 

/* */ 

/* request-id : */ 

/* consists of ( traffic-id , request-no) which uniquely */ 

/* identifies each request being processed by MDBS */ 

19.2 end proc 



end module 
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16.1 proc BROADCAST-ALL-ADDRESS-GENERATION( input : request— id, 

backend-no, cluster-id, output : nothing); 

/* This routine broadcasts the backend number selected for record*/ 

/* insertion to all the backends. */ 

/* */ 

/* request-id : */ 

/* consists of (traffic-id, request-no) which uniquely */ 

/* identifies each request being processed by MDBS */ 

/* backend-no : */ 

/* the number of the backend minicomputer selected to insert */ 

/* a new record into the database store */ 

/* cluster-id : */ 

/* uniquely identifies each cluster */ 

16 .2 end proc 

23.1 proc BROADCAST-ALL-DESCRIPTOR-SEARCH( input : request-id, 

descriptor , descriptor- id, output : nothing); 

/* This routine broadcasts a new descriptor id to all the backends.*/ 
/* */ 
/* request-id : */ 

/* consists of (traffic-id, request-no) which uniquely identifies*/ 
/* each request being processed by MDBS */ 

/* descriptor-id : . */ 

/* uniquely identifies each descriptor */ 

23 .2 end proc 

29. end task 



B.3 Part III - The Post Processing Process 

/* (1) Part III : The Post Processing Process */ 
/* (2) Design : POST-PROCESSING */ 
/* (3) Designers : A. Orooji, Z.Z. Shi, P.R. Strawser */ 
/* (4) Date : Feb. 4, 1982 */ 
/* (5) Modified : Feb. 19, 1982 */ 
/* Apr. 12, 1982 */ 
/* Apr. 26, 1982 */ 
/* Jun. 22, 1982 */ 
/* Jun. 25, 1982 */ 
/* Jul. 8, 1982 */ 
/* (6) Purpose : */ 
/* This is the Post Processing process. It consists of the */ 
/* functions which must be performed before the results of a */ 
/* request or a transaction are forwarded to the host machine. */ 



PAGE 102 



Procedure Hierarchy for POST-PROCESSING 
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Program Structure of POST-PROCESSING (Jackson Chart) 
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Program Structure of PP-CTRL- TASK -MSG (Jackson Chart) 
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Program Structure of PP-BE-TASK-MSG (Jackson Chart) 
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Program Structure of REPLY -MONITOR (Jackson Chart) 
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(10) Data Structures 

/* The data structure definitions are included in the program */ 
/* specifications. */ 



(11) Program Specifications 



1. Task POST-PROCESSING 

/* sender : */ 

/* the sender name of the next message for POST-PROCESSING; */ 

/* the possible values are: 'a task in the controller' and 'a */ 

/* task in a backend' */ 

2. do initialization work; 

3. while 'true' do. /* do forever */ 

4. if. according to the task scheduling this task should release 

the processor 

5. then 

6. release the processor and wait; 

7 . end if 

/* get the next message for POST-PROCESSING */ 

8. perform PP-RECEIVE$MESSAGE; 

/* get the sender name of the next message for POST-PROCESSING */ 

9. perform PP-RECEIVE$SENDER( sender ) ; 

10. case sender value 

11. 'a task in the controller': 

12. perform PP-CTRL-TASK-MSG; 

13. 'a task in a backend': 

14. perform PP-BE-TASK-MSG; 

15. 'otherwise': 

16. system error; 

17 . end case 

18. end while 



12.1 proc PP-CTRL-TASK-MSG ( input : nothing, output : nothing ); 

/* This routine is used when there is a message for POST- */ 
/* PROCESSING from a task in the controller. */ 
/* */ 
/* TrafficUnit : */ 
/* is either a request or a transaction */ 
/* traffic-id : */ 
/* an integer that identifies a traffic unit */ 
/* request-id : */ 
/* consists of (traffic-id, request-no) which uniquely */ 
/* identifies each request being processed by MDBS */ 
/* MessageType : */ 
/* indicates the type of a message (from a task in the */ 
/* controller) for POST-PROCESSING; the possible values are: */ 
/* 'number-of-requests-in-a-traff ic-unit message', 'aggregate-*/ 
/* operators message' and 'requests-with-error message' */ 
/* RequestCountMessagePtr : */ 
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/* a pointer to (traffic-id, request-count) */ 

/* request-count : */ 

/* number of requests in a traffic unit */ 

/* AggregateOperatorsMessagePtr : */ 

/* a pointer to ((request-id, (attribute, */ 

/* aggregate-operator) [,...]) I , . . . ] ) */ 

/* RequestsWithErrorPtr : */ 

/* a pointer to ( (request-id, request , error-message) [,...] ) */ 

/* routing-indicator : */ 

/* indicates the type of the results sent to REPLY -MONITOR; */ 

/* its value is: 'requests-with-error message' */ 

/* The message is from REQUEST-PREPARATION. */ 

/*■ There is a MessageType that indicates the message contains */ 

/* - number of requests in a traffic unit (provided by PARSER)*/ 

/* - error messages (provided by PARSER) */ 

/* - aggregate operators in a retrieve request (provided by */ 

/* PARSER) */ 

12.2 perform PP-RECEIVE$MESSAGE-TYPE(MessageType) ; 

12.3 case MessageType value 

12.4 'number-of-requests-in-a-traff ic-unit message' : 

12.5 perform PP-RECEIVE$REQ-COUNT(RequestCountMessagePtr) ; 



/* save the information to be used by REPLY -MONITOR later */ 

12.6 perform PP-REQUEST-COUNT$SAVE(RequestCountMessagePtr ) ; 

12.7 'aggregate-operators message' : 

12.8 perform PP-RECEIVE$AGGR-OP(AggregateOperatorsMessagePtr ) ; 

/* save the information to be used by AGGREGATE-POST- */ 

/* OPERATION later */ 

12.9 perform PP-AGGR-OP$SAVE(AggregateOperatorsMessagePtr ) ; 

12 . 10 'requests-with-error message' : 

12.11 perform PP-RECEIVE$ERROR-MSG(RequestsWithErrorPtr ) ; 

/* send the error messages to the host machine */ 

12.12 routing-indicator : = 'requests-with-error message'; 

12.13 perform REPLY -MONITOR ( routing-indicator, NULL, 

RequestsWithErrorPtr) ; 

12.14 'otherwise' : 

12.15 system error; 

12.16 end case 

12.17 end proc 



14.1 proc PP-BE-TASK-MSG ( input : nothing, output : nothing ); 

/* This routine is used when there is a message for POST- */ 
/* PROCESSING from a task in a backend. */ 
/* */ 
/* request-id : */ 
/* consists of ( traffic-id , request-no) which uniquely */ 
/* identifies each request being processed by MDBS */ 
/* routing-indicator : */ 
/* indicates where the results should go to; it also */ 
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/* indicates the type of the results; the possible values */ 



/* are: 'REPLY-MONITOR' and 'AGGREGATE-POST-OPERATION' */ 
/* Results : */ 
/* results returned from a backend */ 
/* PartialResults : */ 
/* partial results returned from a backend (for retrieve */ 
/* requests with aggregate operators) */ 
/* last : */ 
/* a flag ('true' or 'false'); it indicates whether or not*/ 
/* all the backends have returned their results */ 
/* AggregateResultsPtr : */ 
/* a pointer to the results computed by AGGREGATE-POST- */ 
/* OPERATION from partial results */ 



/* Get the request-id and the routing-indicator in the message */ 

14.2 perform PP-RECEIVE$ROUTING-INDICATOR( request- id, routing-indicator); 

14.3 case routing-indicator value 

14.4 'REPLY-MONITOR': 

/* receive the results returned by the backend */ 

14.5 perform PP-RECEIVE$RESULTS(Results) : 

/* send the results to the host machine */ 

14.6 perform REPLY -MONITOR(routing-indicator. request-id, Results); 



14.7 

14.8 

14.9 

14.10 

14.11 



14.12 

14.13 



'AGGREGATE-POST-OPERATION' : 

/* receive the partial results returned by the backend */ 
perform PP-RECEIVE$PARTIAL-RESULTS (PartialResults ) ; 
perform AGGREGATE-POST-OPERATION ( request-id , 

PartialResults, AggregateResultsPtr, last); 

if last 
then 

/* The results are ready. */ 

/* Send the results to the host machine. */ 
perform REPLY -M0NIT0R( routing-indicator , request- id, 

AggregateResultsPtr) ; 

end if 



14.14 'otherwise': 

14.15 system error; 

14.16 end case 

14.17 end proc 



12.13.1 proc REPLY-MONITOR( input : routing-indicator , request-id, Results , 

output : nothing); 

/* This routine sends the results to the host machine. It */ 

/* will also send a completion signal to the host machine */ 

/* when all the results have been sent. */ 

/* */ 
/* routing-indicator : */ 

/* indicates the type of results; the possible values are:*/ 
/* 'requests-with-error message', 'REPLY -MONITOR' and */ 

/* 'AGGREGATE-POST-OPERATION' */ 

/* request-id : */ 
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/* consists of (traffic-id, request-no) which uniquely */ 
/* identifies each request being processed by MDBS */ 
/* Results : */ 
/* is one of the following two */ 
/* - the results returned from a backend */ 
/* - a pointer to the results when the results are from*/ 
/* either PARSER or AGGREGATE-POST-OPERATION */ 
/* last : */ 
/* a flag ('true' or 'false'); it indicates whether or not*/ 
/* all the backends have returned their results */ 
/* BufferFull : */ 
/* a flag ('true' or 'false'); it indicates whether or not*/ 
/* the buffer used for storing the results returned by the*/ 
/* backends is full */ 



12.13.2 case routing-indicator value 

12.13.3 'requests-with-error message': 

12.13.4 send the error messages to the host machine; 

/* Results is pointing to the results */ 

12.13 .5 'AGGREGATE-POST-OPERATION' : 

12.13.6 send the results of aggregations to the host machine; 

/* Results is pointing to the results */ 

/* send a completion signal to the host machine */ 

12.13.7 perform SEND-COMPLETION-SIGNAL(request-id) ; 



12.13.8 

12.13.9 

12.13.10 

12.13.11 

12.13.12 

12.13.13 

12.13.14 

12.13.15 



12.13.16 



12.13.17 

12.13.18 



'REPLY-MONITOR' : 

/* store the results returned by the backend in a buffer */ 
perform PP-RESULTS$STORE(request-id , routing-indicator, 

Results, BufferFull, last); 

if BufferFull 
then 

/* the buffer used for storing the results is full */ 
send the results to the host machine; 

end if 
if last 
then 

/* All the backends have returned their results. */ 

/* Send the results remaining in the buffer */ 

/* to the host machine. */ 

send the results to the host machine; 

/* All the results for the request have been sent*/ 

/* to the host machine. */ 

/* Send a completion signal to the host machine. */ 
perform SEND -COMPLETION- SIGNAL( request-id) ; 

end if 



12 .13 .19 'otherwise' : 

12.13.20 system error; 

12.13.21 end case 
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12.13.7.1 proc SEND-COMPLETION-SIGNAL( input : request-id, output : nothing); 



/* This routine sends a request-completion signal to the */ 
/* host machine. It also checks to see whether all the */ 
/* results for a transaction have been sent to the host */ 
/* machine. If so, it sends a transaction-completion signal*/ 
/* to the host machine. */ 
/* */ 
/* request-id : */ 
/* consists of (traffic-id, request-no) which uniquely */ 
/* identifies each request being processed by MDBS */ 
/* Transact ionDone : */ 
/* a flag ('true' or 'false'); it indicates whether or */ 
/* not all the requests in a transaction have been */ 
/* completed */ 
/* NonTransaction : */ 
/* a constant that is assigned to request-no of a */ 
/* request that is not part of a transaction. (We */ 



/* recall that a request is identified by its request-id*/ 

/* which is ( traffic-id, request-no ) . If the request is */ 

/* not part of a transaction, traffic-id identifies the */ 

/* request and request-no can be set to NonTransaction. */ 

/* By doing this, we will be able to tell whether or not*/ 

/* a request is part of a transaction.) */ 

send a request-completion signal to the host machine; 

/* Check to see if the request is part of a transaction */ 
if request-no ~= NonTransaction 
then 

/* The request is part of a transaction. */ 

/* Indicate that one of the requests in the traffic */ 

/* unit has been completed and check to see whether */ 

/* all the results for the traffic unit have been */ 

/* sent to the host machine. */ 

perform PP-REQUEST-COUNT$LAST-REQUEST-CHECK ( traf f ic-id , 

Trans act ionDone) ; 

if TransactionDone 
then 

/* All the results for the traffic unit have */ 

/* been sent to the host machine. */ 

send a transaction-completion signal to the 

host machine; 

end if 

end if 
end proc 

14.9.1 proc AGGREGATE -POST-OPERATION ( input : request-id ,PartialResults , 

output : AggregateResultsPtr , last ); 



/* This routine performs the aggregate operations on the */ 
/* partial results returned by the backends. It will set */ 
/* 'last' to indicate whether or not all the backends have */ 
/* returned their partial results. */ 
/* */ 



12.13.7.2 

12.13.7.3 

12.13.7.4 



12.13.7.5 



12.13.7.6 

12.13.7.7 



12.13.7.8 

12.13.7.9 

12.13.7.10 

12.13.7.11 
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/* request-id : */ 

/* consists of ( traffic-id , request-no) which uniquely */ 

/* identifies each request being processed by MDBS */ 

/* PartialResults : */ 

/* partial results returned from a backend (for retrieve */ 

/* requests with aggregate operators) */ 

/* AggregateResultsPtr : */ 

/* a pointer to the results computed by AGGREGATE-POST- */ 

/* OPERATION from partial results */ 

/* last : */ 

/* a flag ('true' or 'false'); it indicates whether or not*/ 

/* all the backends have returned their results */ 

/* NoBackends : */ 

/* total number of backends in MDBS. (This is a variable */ 

/* defined in SYSGEN.) */ 

14.9 .2 end proc 



module PP- RECEIVE 

programs MESSAGE, SENDER, MESSAGE-TYPE, REQ-CODNT, AGGR-OP , 

ERROR-MSG, ROUTING-INDICATOR, RESULTS, PARTIAL- RESULTS 
datasets MessageBuf f er 

/* used to store messages for POST-PROCESSING */ 

8.1 proc MESSAGE ( input : nothing, output : nothing ); 

/* This routine gets the next message for POST-PROCESSING */ 

/* and stores it in MessageBuf fer . */ 

8.2 end proc 



9.1 proc SENDER ( input : nothing, output : sender ); 

/* This routine returns the sender name of the next message for */ 

/* POST-PROCESSING. */ 

/* */ 

/* sender : */ 

/* the sender name of the next message for POST-PROCESSING; */ 

/* the possible values are: 'a task in the controller' and */ 

/* 'a task in a backend' */ 

9.2 end proc 

12.2.1 proc MESSAGE-TYPE ( input : nothing, output : MessageType ); 

/* This routine returns the message type of the next message */ 
/* (from a task in the controller) for POST-PROCESSING. */ 

/* */ 

/* MessageType : */ 

/* indicates the type of a message (from a task in the */ 

/* controller) for POST-PROCESSING; the possible values are: */ 

/* 'number-of-requests-in-a-traf f ic-unit message', */ 

/* 'aggregate-operators message' and 'requests-with-error */ 

/* message' */ 

12.2.2 end proc 

12.5.1 proc REQ-COUNT ( input : nothing, output : RequestCountMessagePtr ); 
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/* This routine returns RequestCountMessagePtr sent by REQUEST- */ 

/* PREPARATION. */ 

/* */ 

/* RequestCountMessagePtr : */ 

/* a pointer to (traffic-id, request-count) */ 

/* request-count : */ 

/* number of requests in a traffic unit */ 

12.5.2 end proc 

12.8.1 proc AGGR-OP ( input : nothing, output : AggregateOperatorsMessagePtr); 

/* This routine returns AggregateOperatorsMessagePtr sent by */ 

/* REQUEST-PREPARATION. */ 

/* */ 

/* AggregateOperatorsMessagePtr : */ 

/* a pointer to ((request- id, (attribute, */ 

/* aggregate-operator )(,...]) I ,...]) */ 

12.8.2 end proc 

12.11.1 proc ERROR-MSG ( input : nothing, output : RequestsWithErrorPtr) ; 

/* This routine returns RequestsWithErrorPtr sent by REQUEST- */ 

/* PREPARATION. */ 

/* */ 

/* RequestsWithErrorPtr : */ 

/* a pointer to ( (request-id , request , error-message ) [,...] ) */ 

12.11.2 end proc 



14.2.1 proc ROUTING-INDICATOR ( input : nothing, 

output : request-id , routing-indicator ); 

/* This routine returns the request-id and the routing-indicator*/ 



/* in the next message (from a backend) for POST-PROCESSING. */ 

/* */ 

/* request-id : */ 

/* consists of (traff ic-id , request-no) which uniquely */ 

/* identifies each request being processed by MDBS */ 

/* routing-indicator : */ 

/* indicates where the results should go to; it also */ 

/* indicates the type of the results; the possible values */ 

/* are: 'REPLY-MONITOR' and 'AGGREGATE-POST-OPERATION' */ 

14.2.2 end proc 

14.5.1 proc RESULTS ( input : nothing, output : Results ); 

/* This routine returns the results sent by a backend. */ 

/* */ 

/* Results : */ 

/* results returned from a backend */ 

14.5.2 end proc 

14.8.1 proc PARTIAL-RESULTS ( input : nothing, output : PartialResults) ; 

/* This routine returns the partial results sent by a backend. */ 
/* */ 

/* PartialResults : */ 

/* partial results returned from a backend */ 

14.8.2 end proc 
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end module 



module PP-REQUEST-COUNT 

programs SAVE, LAST-REQUEST-CHECK 

datasets CountBuffer /* used to save RequestCountMessagePtr's */ 

12.6 .1 proc SAVE ( input : RequestCountMessagePtr , output : nothing ); 

/* This routine saves RequestCountMessagePtr to be used by */ 

/* LAST-REQUEST-CHECK later. */ 

/* */ 

/* RequestCountMessagePtr : */ 

/* a pointer to (traffic-id, request-count) */ 

/* request-count : */ 

/* number of requests in a traffic unit */ 

12.6.2 end proc 

12.13.7.5.1 proc LAST-REQUEST-CHECK ( input : traffic-id, 

output : TransactionDone) ; 

/* This routine remembers that one of the requests in the */ 
/* traffic unit has been completed. It will set */ 

/* 'TransactionDone' to indicate whether or not all the */ 
/* requests in the traffic unit have been completed. */ 

/* */ 
/* traffic-id : . */ 

/* an integer that identifies a traffic unit */ 

/* TransactionDone : */ 

/* a .flag ('true' or 'false'); it indicates whether or */ 

/* not all the requests in a traffic unit have been */ 

/* completed */ 

12.13.7.5.2 remember that one of the requests in the traffic unit has 

been completed; 

12.13.7.5.3 if_ all the requests in the traffic unit have been completed 

12.13.7.5.4 then 

12.13.7.5.5 TransactionDone := 'true'; 

12.13.7.5.6 free the space used to store the number of requests 

in the traffic unit; 

12.13.7.5.7 else 

12.13.7.5.8 TransactionDone := 'false'; 

12.13.7.5.9 end if 

12.13.7.5.10 end proc 

end module 



module PP-AGGR-OP 
programs SAVE 

/* there will be other procedure(s) in this module, e.g., a */ 
/* procedure that returns the aggregate operators for a request*/ 
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datasets AggregateBuf f er 

/* used to save AggregateOperatorsMessagePtr 's */ 

12.9.1 proc SAVE ( input : AggregateOperatorsMessagePtr, output : nothing ); 

/* This routine saves AggregateOperatorsMessagePtr to be used */ 



/* by AGGREGATE-POST-OPERATION later. */ 

jic k j 

/* AggregateOperatorsMessagePtr : */ 

/* a pointer to ( (request-id , (attribute , */ 

/* aggregate-operator) [,...]) L ,...] ) */ 

12.9.2 end proc 



end module 



module PP-RESULTS 
programs STORE 

/* there will be other procedure(s) in this module, e.g., a */ 

/* procedure that returns the results stored in ResultsBuff er*/ 
datasets ResultsBuff er 

/* used to store results returned by backends */ 

12.13.9.1 proc STORE ( input : request-id , routing-indicator , Results , 

output : Buff erFull , last ); 

/* This routine stores the results returned by a backend in */ 
/* a buffer (the results will be used by REPLY -MONITOR , */ 

/* later). It will set 'Buff erFull' to indicate whether or */ 
/* not the buffer used for storing the results returned by */ 



/* the backends is full. It will also set 'lastT to */ 
/* indicate whether or not all the backends have returned */ 
/* their results. */ 
/* */ 
/* request-id : */ 
/* consists of ( traffic-id , request-no) which uniquely */ 
/* identifies each request being processed by MDBS */ 
/* routing-indicator : */ 
/* indicates the type of results */ 
/* Results : */ 
/* results retuned from a backend */ 
/* BufferFull : */ 
/* a flag ('true' or 'false'); it indicates whether or */ 
/* not the buffer used for storing the results returned */ 
/* by the backends is full */ 
/* last : */ 
/* a flag ('true' or 'false'); it indicates whether or */ 
/* not all the backends have returned their results */ 
/* NoBackends : */ 
/* total number of backends in MDBS. (This is a variable */ 
/* defined in SYSGEN.) */ 



12.13.9.2 

12.13.9.3 



store the results into ResultsBuf f er ; 
set the flag 'BufferFull'; 
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12.13.9.4 set the flag 'last'; 

12.13.9.5 end proc 

end module 



19. end task 
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APPENDIX C 

THE SSL SPECIFICATION FOR RECORD PROCESSING 



The SSL specification for record processing is given in this appendix. 
The specification consists of five parts: a control subfunction of record 

processing, a retrieve processing subfunction, an insert processing subfuc- 
tion, a delete processing subfunction and an update processing subfunction. 
They are specified in Parts I, II, III, IV and V respectively. 

C.l Part I -The Control Subfunction of Record Processing 

/* (1) Part I : The control subfunction of Record Processing */ 

/* (2) Design : RECORD PROCESSING */ 

/* (3) Designers : He Xingui, Masanobu Higashida */ 

/* (4) Date : Jan. 28, 1982 */ 

/* (5) Modified : Feb. 1, 1982 */ 

/* Feb. 18, 1982 */ 

/* Ma-r. 11, 1982 */ 

/* April 1, 1982 */ 

/* April 9, 1982 */ 

/* April 15,1982 */ 

/* April 27,1982 */ 

/* May 17, 1982 */ 

/* May 19, 1982 */ 

/* (6) Purpose : */ 

/* The control subfunction of Record Processng is designed for */ 

/* analyzing the information coming from Directory Management to decide*/ 

/* what request processing subfunction should be executed, and then to */ 

/* transfer control to the relevant procedure. */ 

/* (7) Input: */ 

/* Input consists of a formated request and a set of disk */ 

/* addresses where the relevant data are stored. */ 
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/* (8) Procedure Hierarchy for The Control Subfunction 



*/ 
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/* (9) Jackson Chart: 
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/* (11) Program Specifications */ 

1 task RECORD_PROCES SING; 

/* This task is to be used to analyze a request and then execute */ 
/* the relevant procedure. */ 

2 list REQUEST: string; 

3 set ADDRESSES : integer; 

4 scalar NewTrack: logical; 

5 while 'true' do. /* Do forever. */ 

6 perform GET_REQ_ADD_NEW(REQUEST, ADDRESSES, NewTrack); 

/* Get a messageC a request REQUEST, */ 
/* a set of addresses ADDRESSES */ 

/* and new track indicator NewTrack) */ 
/* from a queue. */ 

7 case REQUEST. REQUEST_TYPE value 

8 'RETRIEVE' : 

perform RETRIEVE_PROCESSING( REQUEST. QUERY, 

REQUEST . TARGET , ADDRESSES ) ; 

9 'UPDATE' : 

perform UPDATE_PROCESSING( REQUEST. QUERY, ADDRESSES, 

REQUEST. attribute, REQUEST. valuel) ; 

10 'DELETE': 

perform DELETE_PROCESSING( REQUEST. QUERY, ADDRESSES); 

11 'INSERT' : 

perform INSERT_PROCESSING( REQUEST. RECORD , ADDRESSES , NewTrack) ; 

12 end case ; 

*/ 



13 perform SEND_MESSAGE /* Send completion signal to CONTROLLER. 

14 end while : 



PAGE 120 



C . 2 Part II-The Retrieve Processing Subfunction 



/* 


(1) 


Part II 


: Retrieve Processing Subfunction 


*/ 


/* 


(2) 


Design 


: RECORD PROCESSING 


*/ 


/* 


(3) 


Designers 


: He Xingui, Masanobu Higashida 


*/ 


/* 


(4) 


Date 


: Jan. 28, 1982 


*/ 


/* 


(5) 


Modified 


: Feb. 1, 1982 


*/ 


/* 






Feb. 18, 1982 


*/ 


/* 






Mar. 11, 1982 


*/ 


/* 






April 1, 1982 


*/ 


/* 






April 9, 1982 


*/ 


/* 






April 15,1982 


*/ 


/* 






April 27,1982 


*/ 


/* 






May 17, 1982 


*/ 


/* 






May 19, 1982 


*/ 


/* 


(6) 


Purpose 


: 


*/ 


/* 




The procedure is designed for retrieving the records 


*/ 


/* 


satisfying the 


query in the request. 


*/ 
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/* (8) Procedure Hierarchy for Retrieve Processing Subfunction */ 
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/* (9) Jackson Chart: 
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Check whether the 
record satisfies 
the query 

H _ + 
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/* (11) Program Specifications */ 

8.1 proc RETRIEVE_PROCESSING( input : QUERY, TARGET, ADDRES SES ) ; 

/* This procedure is to be used for processing of RETRIEVE request. */ 

8.2 list QUERY, TARGET, resul t : string; 

8.3 set ADDRESSES: integer; 

8.4 array TRACKJBUFFER , RE SUT JBUFFER : word; 

8.5 scalar indexA, indexB: integer; 

/* these are pointers for ADDRESSES and TRACKJBUFFER respectively */ 

8.6 scalar satisfied, ok : logical; 

8.7 scalar sum, count , max, min: real; 



8.8 perform GET BUFFER (RESULT BUFFER); /* Get RESULT JBUFFER. */ 

/* Initialize the partial results. */ 

8.9 sum: =0 ; 

8.10 count :=0; 

8.11 max:=the smallest number; 

8.12 min:=the largest number; 



/* Process data track by track . */ 

8.13 for each address ADDRES SES ( indexA) in ADDRESSES do. 



/* Fetch one track into TRACKJBUFFER . */ 

8.14 perform FETCH JT0_TRACK JBUFFER ( indexA , TRACK JBUF FER ) ; 

/* Select records in TRACK_BUFFER one by one. */ 

8.15 for each record TRACK_BUFFER( indexB) in TRACKJBUFFER do 

8.16 iE the record is not marked a 'deletion flag' 

8.17 then 

/* Check whether the record satisfies the QUERY. */ 

8.18 perform CHECK_QUERY( QUERY, TRACKJBUFFER, indexB, satisfied) ; 



8.19 

8.20 
8.21 
8.22 

8.23 

8.24 

8.25 



if satisf ied='true' 
then 

if there is aggregate operation 
then 

/* Compute partial results and count. */ 

perform AGGR£ATE_OPERATION ( TRACKJBUFFER , indexB , 

sum, coun t , max, min, TARGET) ; 

else 

/* Get result by projection. */ 

perform PROJECT (TRACKJBUFFER, indexB, result , TARGET) ; 



/* Collect it into RESULT JBUFFER. */ 

8.26 perform STUFF JBUFFER ( RE SUL T_BUF FER , 

result , length_of_result) ; 

8.27 end if ; 

8.28 end if : 

8.29 end if ; 

8.30 end for ; /* indexB */ 

8.31 end for : /* indexA */ 
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/* Stuff the partial results into RESULT_BUFFER, if any. */ 

8.32 perform STUFF_BUFFER(RESULT_BUFFER, sum, length_of_sum) ; 

8.33 perform STUFF_BUFFER(RESULT_BUFFER, count, length_of_count) ; 

8.34 perform STUFF_BUFFER(RESULT_BUFFER, max, length_of_max) ; 

8.35 perform STUFF_BUFFER(RESULT_BUFFER, min, length_of_min) ; 

/* Send the collected results in RESULT_BUFFER to CONTROLLER. */ 

8.36 perform FLUSH BUFFER( RESULT BUFFER, ok ); 

8.37 end proc ; 



8.18.1 £roc CHECK_QUERY ( input : QUERY, TRACK_BUFFER , indexB, output : satisfied) ; 



/* This procedure is used to check whether the record in */ 

/* TRACK_BUFFER( indexB) satisfies the QUERY or not. */ 

8.18.2 list QUERY: string; 

8.18.3 array TRACK_BUFFER : word; 

8.18.4 scalar indexB, indexC , indexP : integer; 

/* these are pointers for TRACK_BUFFER , QUERY. CONJUNCTION */ 

/* and QUERY. CONJUNCTION. PREDICATE respectively. */ 

8.18.5 scalar satisfied: logical; 

/* Check whether the record satisfies the QUERY */ 

8.18.6 for each conjunction QUERY ( indexC ,*) in QUERY do 

satisf ied='true' ; 

/* Check whether the record satisfies the conjunction */ 

/* pointed by indexC. */ 



8.18.7 for each predicate QUERY ( indexC , indexP) in QUERY ( indexC ,*) do. 

8.18.8 if. The record in TRACK_BUFFER( indexB) does not satisfy 

the predicate in QUERY (indexC, indexP ) ; 

8.18.9 then 

8.18.10 satisf ied :='false' ; 

8.18.11 exit the loop; 

8.18.12 end if ; 

8.18.13 end for; /* indexP */ 

8.18.14 if satisf ied true' then 

8.18.15 return ; 

8.18.16 end if ; 

8.18.17 end for ; 

8.18.18 end proc ; 



/ *indexC */ 
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C.3 part IIl-The Insert Processing Subfunction 



/* 


(1) 


Part III 


: Insert Processing Subfunction 


*/ 


/* 


(2) 


Design 


: RECORD PROCESSING 


*/ 


/* 


(3) 


Designers 


: He Xingui, Masanobu Higasbida 


*/ 


/* 


(4) 


Date 


: Jan. 28, 1982 


*/ 


/* 


(5) 


Modified 


: Feb. 1, 1982 


*/ 


/* 






Feb. 18, 1982 


*/ 


/* 






Mar. 11 , 1982 


*/ 


/* 






April 1, 1982 


*/ 


/* 






April 9, 1982 


*/ 


/* 






April 15,1982 


*/ 


/* 






April 27,1982 


*/ 


/* 






May 17, 1982 


*/ 


/* 






May 19, 1982 


*/ 


/* 


(6) 


Purpose 




*i 


/* 




The procedure is designed for inserting a record 


into the disk*/ 


/* 


indicated by a 


address . 


*/ 
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(8) Procedure Hierarchy for Insert Processing Subfunction */ 



Insert processing Subfunction 



-i 

1 






1 

FETCH TO TRACK BUFFER 


INSERT RECORD 


STORE TRACK BUFFER 
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/* (9) Jackson Chart: 



*/ 



+ + 

Insert 
request 
processing 
+ + 



+ + 




I Should it insert I 
I into a new track! 



-+ +- 



Insert record 
into TRACK 
BUFFER “ 



No 



I 



Tot 

— + 



+ + 

I Yes lol 

+ ”l + 



Fetch one 
track to 
TRACK BUFFER 



+-+ + +-+ 

(Do nothing I 



+ + 

I Store TRACK BUFFER 1 
I back to disk* 

H + 
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/* (11) Program Specifications */ 

11.1 proc INSERT PROCESSING( input : RECORD , ADDRESS ,NewTrack) ; 

/* This procedure is used for inserting the record into TRACK_BUFFER */ 

/* according to ADDRESS. */ 

11.2 set ADDRESS: integer; 

11.3 array TRACK BUFFER: word; 

11.4 scalar RECOKD: string; 

11.5 scalar NewTrack: logical; 

11.6 if NewTrack='false' 

11 .7 Then 

J T* Fetch one track indicated by ADDRESS to TRACK BUFFER. */ 

11.8 perform FETCH TO TRACK BUFFER(ADDRESS .TRACK BUFFlH): 

11.9 end if : _ _ _ _ 

/* Insert the record into the TRACK BUFFER. */ 

11.10 perform INSERT_RECORD ( RECORD, TRACK JBtJFFER ) ; 

/* Store TRACK BUFFER back to the disk according to ADDRESS. */ 

11.11 perform STORE THACK BUFFER(ADDRESS .TRACK BUFFER) ; 

11.12 end proc : — “ ” 



C.4 Part IV- The Update processing Subfunction 



/* 

/* 

/* 

/* 

/* 

/* 

/* 

/* 

/* 

(* 

/* 

/* 

/* 

/.* 



;i; 

, 2 ) 

3 

4) 

.5) 



Part IV 
Design 
Designers 
Date 

Modified 



(6) Purpose 



>date processing Subfunction 
LCORD PROCESSING 
He Xingui, Masanobu Higashida 
Jan. 2F 



Feb. 1. 

Feb. 18, 

Mar. 11, 

April 1, 

April 9, 

April 15,1982 
April 27,1982 
May 17, 1982 
May 19, 1982 



i 982 

1982 

1982 

1982 

1982 

1982 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



The procedure is designed for implementing the update request */ 
/* with the modifier of type 0, type I or type II. the request with */ 
/* modifier of type III or IV is implemented as a retrieve followed by */ 
/* another type 0 update. */ 
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/* (8) Procedure Hierarchy for Update Processing Subfunction */ 



Update Processing Subfunction 
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FETCH TO CHECK QUERY 

TRACOUFFER “ 



UPDATE 



STORE FLUSH BUFFER 
TRACK - “ 

BUFFER" 





+ 


+ + 


+ 

1 


1 

GET ATTRIBUTE 
VALFE 


UPDATE_KEC0RD 


CHECK_CLU SETR STUFF_BUFFER 


1 

DELETE 

RECORD 
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/* (9) Jackson Chart: 



*/ 
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record satisfies +- 
the query 



H — f- H — h 

I Satisfied processing lol IWhen not satisfied |o| 

H + + + 

I I I 






Compute 

valuel 



Update the 
record in 
TRACK BUFFER 



Check if change 
cluster or not 



I 



It needs to |o 

change cluster 



-+ +- 



It does not need |o 
to change cluster +- 



Collect the 
record into 
RESULT BUFFER 



-+ + H 

Delete original 
record in the 
TRACK BUFFER 
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/* (11) Program Specifications 



*/ 



9.1 proc UPDATE PROCESSING( input ; QUERY, ADDRESSES, attribute, valuel) ; 

/* This procedure is to be used for processing of UPDATE request. */ 

9.2 list QUERY: string; 

9.3 set ADDRESSES: integer; 

9.4 array TRACK_BUFFER , RESULT_BUFFER : word; 

9.5 scalar attribute .valuel : string; 

9.6 scalar indexA indexB: integer; 

/* These are pointers for ADDRESSES and TRACK_BUFFER */ 

/* respectively. */ 

9.7 scalar satisfied, ok: logical; 

9.8 perform GET BUFF ER( RESULT BUFFER): /* Get RESULT_BUFFER. */ 

/* Process data track by track . */ 

9.9 for each address ADDRESSES ( indexA) in ADDRESSES do. 

/* Fetch one track into TRACK_BUFFER . */ 

9.10 perform FETCH_TO_TRACK_BUFFER( indexA, TRACK_BUFFER) ; 

/* Select records in TRACK_BUFFER one by one. */ 

9.11 for each record TRACK_BUFFER( indexB) in TRACK_BUFFER do 

9.12 if. the record is not marked a 'deletion flag' 

9.13 then 



9.14 



/* Check whether the record satisfies the QUERY. 
perform CHE CK_QUERY( QUERY, TRACK_BUFFER, indexB, satisfied) ; 



*/ 



9.15 

9.16 



if satisfied='true' 

then /* Update the retrieved record in 



*/ 

*/ 

*/ 

*/ 



/* TRACK_BUFFER( indexB) and collect it into 
/* RESULT_BUFFER , or store it back to 
/* the original place. 



9.18 

9.19 

9.20 



9.17 




/* Store TRACK_BUFFER back to disk. 

9.21 perform STORE_TRACK_BUFFER( indexA, TRACK_BUFFER) ; 

9.22 end for: /* indexA */ 



*/ 



/* Send the collected results in RESULT_BUFFER to CONTROLLER. */ 

9.23 perform FLUSH_BUFFER(RESULT_BUFFER, ok); 

9.24 end proc : 
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9.17.1 proc UPDATE (indexB, attribute, valuel ,TRACK_BUFFER,RESULT_BUFFER) ; 

/* This procedure updates the record in TRACK_BUFFER( indexB) . */ 

9.17.2 array TRACK_BUFFER , RESULT_BUFFER : word ; 

9.17.3 scalar attribute .value .valuel : string; 

9.17.4 scalar indexB: integer; /* This is a pionter for TRACK_BUFFER */ 

9.17.5 scalar check result: logical; 

/* Get the attribute value of the record in TRACK_BUFFER( indexB) .*/ 

9.17.6 perform GET_ATTRIBUTE_VALUE( indexB , attribute .value) ; 

/* Here f(...) is a function procedure. 

9.17.7 valuel=f (value) ; 

/* Update the value of attribute in TRACK_BUFFER( indexB) 

/* into valuel . 

perform UPDATE_RECORD (attribute , valuel ,TRACK_BUFFER, indexB) ; 

/* Check if the record needs to change cluster or not. 
perform CHECK_CLUSTER(TRACK_BUFFER, indexB ,cluster_changed) ; 

if cluster_changed='true' 

then /* The updated record needs to change cluster. 

I* Collect the record into RESULT_BUFFER. 
perform STUFF_BUFFER(RESULT_BUFFER,TRACK_BUFFER( indexB) , 

length_of_the_record) ; 

/* Delete the original record in TRACK_BUFFER . */ 

perform DELETE_RECORD ( TRACK_BUFFER , indexB ) ; 
end if ; 

.17.15 end proc : 



9.17.8 

9.17.9 

9.17.10 

9.17.11 

9.17.12 

9.17.13 

9.17.14 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 
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C.5 Part V- The Delete Process ing Subfunction 
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*/ 
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*/ 
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/* 
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*/ 


/* 
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/* 
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request . 


*/ 
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/* (8) Procedure Hierarchy for Delete Processing Subfunction */ 
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+ + + + 

FETCH TO CHECK QUERY DELETE STORE TRACK BUFFER 
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/* (11) Program Specifications */ 

10.1 proc DELETE PROCESSING( input: QUERY, ADDRESSES) ; 

/* This procedure is to be used for processing of DELETE request. */ 



10.2 

10.3 

10.4 

10.5 

10.6 



10.7 



10.8 



10.9 

10.10 
10.11 



10.12 

10.13 

10.14 

10.15 

10.16 

10.17 

10.18 

10.19 

10.20 
10.21 



list QUERY: string; 
set ADDRESSES: integer; 
array TRACK_BUFFER : word; 
scalar indexA, indexB : integer; 

/★These are pointers for ADDRESSES and TRACK_BUFFER respectively */ 
scalar satisfied: logical; 

/* Process data track by track . */ 

for each address ADDRESSESC indexA) in ADDRESSES do. 

/* Fetch one track into TRACK_BUFFER . */ 

perform FETCH_TO_TRACK_BUFFER( indexA, TRACK_BUFFER) ; 

/* Select records in TRACK_BUFFER one by one. */ 

for each record TRACK_BUFFER( indexB) in TRACK_BUFFER do. 
if the record is not marked a 'deletion flag' 
then 

/* Check whether the record satisfies the QUERY. . */ 



perform CHECK_QUERY ( QUERY , TRACK_BUFFER , indexB, satisfied) ; 
if satisf ied='true' 

then /* Mark the retrieved record in TRACK_BUFFER( indexB) .*/ 
perform DELETE ( TRACK_BUFFER , indexB ) ; 
end if : 
end if ; 

end for ; /* indexB */ 

perform STORE_TRACK_BUFFER( indexA, TRACK_BUFFER) ; 

/* Store TRACK_BUFFER back to disk. */ 

end for : /* indexA */ 

end proc ; 
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APPENDIX D 

TEE SSL SPECIFICATION FOR THE TEST REQUEST GENERATION AND EXECUTION PACKAGE 

The program specification for the test request generation and execution 
package is shown in this appendix. The specification design is composed of 
two parts. Part one includes the top level design. Part two includes the 
module concerned with the handling of the output from a test. 



D.l Part I - The Top Level of Test Request Generation and Execution 



/* (1) Part I - The Top Level of Test Request Generation and */ 
/* Execution */ 
/* (2) Design: MDBS TEST */ 
/* (3) Designer: D.S. Kerr */ 
/* (4) Date: July 8, 1982 */ 
/* */ 
/* (6) Purpose: */ 



This program can be used to test and demonstrate MDBS. The 
execution of this program is called a session. Each session can be 
divided into any number of subsessions. During a subsession the user 
can do one of the following: 

(A) Execute a list of requests that was previously 
stored in a file. 

(B) Prompt the user for a list of requests to be 
stored in a file for later use. 

(C) Retrieve a list of requests that were previously 
stored in a file and then allow the user to select 
requests from that list for execution. This selection can 
be done in any order. The user will also be able to enter 
a new request to be executed. 

(D) Modify an existing list of requests that was 
previously stored in a file. 

In this version, requests are executed one at a time. A request 
is sent to MDBS. Then the program waits for a response before sending 
the next request. Transactions are not allowed. 

Output may be directed to the user's terminal or to a file or to 



both 



PAGE 138 



(8) Procedure Hierarchy for MDBS Test 
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NEW LIST SUB 



MODIFY SUB 
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SELECT SUB 
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•H +• 



■H +■ 



DETERMINE_INPUT_FILE 

DISPLAY 



OUTM$ 



EXECUTE 

GET_NEW_REQUEST 



+- 

I 



OLD LIST SUB 






DETERMINE INPUT FILE 



OUTM$ 



EXECUTE 



ENTER AND SAVE 
REQUESTS “ ~ 

NEW REQUEST SUB 



INSERT SUB UPDATE SUB DELETE SUB RETRIEVE SUB 
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(10) Data Structures 



The data structures definitions are included at the beginning of each 
procedure definition in (11) below. 



(11) Program Specifications 



1. task MDBS Test; 

2. scalar more-subsessions; /* flag: TRUE - continue, FALSE - stop */ 

3. Print initial message to user; 

4. more-subsessions := TRUE; 

5. while more-subsessions d£ 

6. perform SUBSESSION; 

7. Prompt for continue message; 

8. Read continue message; 

9. .if. user does not want to continue 

10 . then 

11. more-subsessions := FALSE; 

12. end if 

13 . end while ; 

14. end task : 



6. procedure SUBSESSION; 



/* During a subsession the user is able */ 

/* to generate a group of requests. (NEW_LIST) */ 

/* to modify an old list of requests. (MODIFY) */ 

/* to select requests, one at a time from a list */ 

/* of requests. (SELECT) */ 

/* to run a group of requests. (0LD_LIST) */ 

6.1 scalar current-request-file; /* The name of the file */ 

/* Initial value should be NULL. This name must be */ 

/* retained from one subsession to the next. */ 

6.2 scalar type-of-subsession; /* Possible values are NEW_LIST, 

MODIFY, SELECT and 0LD_LIST */ 



6.3 Prompt for next type-of-subsession; 

6.4 Read next type-of-subsession; 

6.5 case type-of-subsession value 

6.6 NEW_LIST: /* Enter a new request-list */ 

perform NEW_LIST_SUB( current-request-f lie) ; 

6.7 MODIFY: /* Modify an old list */ 

perform MODIFY SUB( current-request-file ); 

6.8 SELECT: /* Select requests, one at a time, from an */ 

/* existing request-list */ 

perform SELECT_SUB( current-request-file ); 
0LD_LIST: /* Execute an existing request-list */ 
perform OLD_LIST_SUB( current-request-file); 



6.9 
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6.10 

6.11 

6.12 



6 . 6.1 

6 . 6.2 



6.6.3 

6 .6 .4 
6.6.5 



6 . 6.6 

6.6.7 

6 . 6.8 

6.6.9 

6 . 6.10 
6 . 6.11 
6 . 6.12 



6.7.1 

6.7.2 



6.7.3 

6.7.4 

6.7.5 



otherwise: Print error message; 
end case ; 
end procedure ; 



procedure NEW_LIST_SUB( output : current-request-f lie ); 
scalar current-request-file; /* name of the file */ 

/* Asks user for requests - one at a time. */ 

/* Saves list of requests in a file with file-name given by */ 
/* user. */ 

scalar request-list-file-name; 

/* of file to use to store the requests */ 
record request; 
scalar next-step; 

/* I(nsert), R(etrieve), U(pdate), D(elete) or F(inish) */ 

Prompt for request-list-file-name; 

Read request-list-file-name; 

Open file( request-list-file-name ) output; 
perform ENTER_AND_SAVE_REQUESTS( request-list-file-name ); 
Close file( request-list-file-name ); 
current-request-file : = request-list-f ile-name ; 
end procedure ; 



procedure MODIFY SUB( input /output : current-request-file ); 
scalar current-request-file; /* The name of the file */ 

/* Retrieve an old request-list and then allow the user to */ 
/* modify it. Requests are examined one at a time allowing */ 



/* changes to be made to each request in turn. A change */ 
/* can be */ 
/* add new request before this one. */ 
/* modify this request. */ 
/* remove this request. */ 
/* make no changes to this request. */ 
/* Note that we must have a way to append new requests at */ 
/* the end of the input request list. */ 
/* */ 
/* The input file ( called input-request-file ) may be */ 
/* either the current-request-file or a different existing */ 
/* request file. */ 
/* */ 
/* The output file ( called new-request-f ile ) may be */ 
/* either the next version of the input-request-file or a */ 
/* new file. */ 



scalar input-request-file; /* The list of requests to be modified. */ 
scalar new-request-f ile ; /* The new list of requests. */ 
scalar next-version; /* flag: TRUE - set new-request-f ile to next */ 
/* version of input-request-file, FALSE - get new name. */ 
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6.7.6 

6.7.7 

6.7.8 

6.7.9 

6.7.10 



6.7.11 

6.7.12 



6.7.13 

6.7.14 

6.7.15 

6.7.16 

6.7.17 

6.7.18 

6.7.19 

6.7.20 

6.7.21 

6.7.22 

6.7.23 

6.7.24 

6.7.25 

6.7.26 

6.7.27 

6.7.28 

6.7.29 

6.7.30 

6.7.31 

6.7.32 

6.7.33 

6.7.34 

6.7.35 

6.7.36 

6.7.37 

6.7.38 

6.7.39 



record request; 

scalar more-requests-in-input-request-f lie ; /* continuation flag */ 
scalar more-requests-to-enter ; /* continuation flag */ 
scalar change-type; /* ADD, MODIFY, REMOVE, or NOCHANGE */ 

scalar next-step; 

/* I(nsert) , R(etrieve), U(pdate), D(elete) or F(inish) */ 

/* Determine input-request-file to be modified. */ 
perform DETERMINE_INPUT_FILE( current-request-file, 

input-request-f lie ); 
open file( input-request-file ) input; 

/* Determine if user wants the name of the new-request-f ile to */ 
/* be the next version of the input-request-file or a new name.*/ 
Prompt user to determine next-version; 

Read next-version; 
if next-version 
then 

Set new-request-f ile to next version of 
input-request-file; 

else 

Prompt for new-request-f ile name; 

Read name of new-request-f ile ; 

end if ; 

open file( new-request-file ) output; 

Read first request from input-request-file; 
more-requests-in-input-request-f ile := TRUE; 

while more-requests-in-input-request-file do 

Prompt user for change-type for this request; 

Read change-type; 
case change- type value 

ADD: /* enter and save the next request */ 
perform GET_NEW_REQUEST( request ); 

Write request into new-request-file; 

MODIFY : 

Prompt and get modified request from user; 

Write new request into new-request-file; 

Read next request from input-request-file; 

REMOVE: 

Read next request from input-request-file; 

NOCHANGE: 

Write current request into new-request-file; 

Read next request from input-request-file; 
otherwise : Print system error message; 
end case ; 
end while : 

/* Note that at this point all the old requests have been */ 

/* processed. However it is possible that the user wants to */ 

/* append more requests. */ 

Prompt user that input file has been processed, but that more 
requests may still be appended; 



6.7.40 
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6.7.41 

6.7.42 
6.7 443 

6.7.44 

6.7.45 


perform ENTER AND SAVE REQUESTS( new-request-f lie ); 
close file( input-request-f ile ); 
close file( new-request-f ile ); 
current-request-f ile := new-request-f ile; 
end procedure; 


6.8.1 

6.8.2 


procedure SELECT SUB( input/output : current-request-f ile ); 
scalar current-request-file; /* The name of the file */ 

/* Retrieve an old list of requests. */ 
/* Allow user to select from this list. */ 
/* Also allow user to enter new request. */ 


6.8.3 

6.8.4 


scalar input-request-file; /* The file containing the requests. */ 
array reauests( MAX NUMBER OF REOUESTS ); 

/* from input-request-file */ 


6.8.5 


scalar number-of-requests ; /* The actual number in */ 
/* input-request-file must be less than */ 

/* MAX_NUMB ER_0F_RE QUESTS */ 


6.8.6 

6.8.7 

6.8.8 


scalar request-number; /* of the request chosen */ 
record new-request; /* Provided by user. */ 
record response; /* to the request being executed. */ 


6.8.9 

6.8.10 


scalar more-to-execute ; /* flag to control loop */ 

scalar next-operation; /* Values can be REQUEST NUMBER, DISPLAY, */ 
/* NEW_REQUEST or STOP */ 


6.8.11 


/* Determine the new input-request-file to use for */ 
/* this subsession. */ 

perform DETERMINE INPUT FILE( current-reauest-f ile . 

input-request-file ) ; 


6.8.12 

6.8.13 


open( input-request-file ); 

Read and store input-request-file into requests checking that 

number-of-requests is less than MAX_NUMBER_OF_RE QUESTS ; 


6.8.14 

6.8.15 


close( input-request-file ); 
perform DISPLAY( requests ); 


6.8.16 

6.8.17 


/* Determine whether response is to go to CRT, file or both. */ 
perform 0UTM$F0RMAT ; 
more-to-execute := TRUE; 


6.8.18 

6.8.19 


while more-to-execute do 

Prompt user for next-operation /* It should be either a */ 
/* request-number, a request-to-display or a */ 

/* new-request */ 


6.8.20 

6.8.21 

6.8.22 


Read next-operation; 
case next-operation value 
REQUEST_NUMBER: 

Check that request-number is less than 
number-of-requests ; 
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6.8.23 



6.8.24 

6.8.25 

6.8.26 

6.8.27 



6.8.28 

6.8.29 

6.8.30 

6.8.31 

6.8.32 

6.8.33 

6.8.34 

6.8.35 



6.9.1 

6 . 9.2 



6.9.3 

6.9.4 

6.9.5 



6.9.6 

6.9.7 

6.9.8 



6.9.9 

6.9.10 

6.9.11 



6.9.12 

6.9.13 

6.9.14 

6.9.15 



perform EXECUTE ( requests ( request-number) , 
response ) ; 

/* Output the response to CRT, file or CRT&file, 
as appropriate. */ 
perform 0PTM$RESP0NSE( response ); 

DISPLAY: perform DISPLAY( requests ); 

NEW_RE QUEST: 

perform GET_NEW_REQUEST( new-request ) ; 
perform EXECUTE( new-request, response ); 

/* Output the response to CRT, file or CRT&file, 
as appropriate. */ 
perform OUTM$RESPONSE( response ); 

STOP: more-to-execute := FALSE; 
otherwise : print error message; 
end case ; 
end while ; 

perform 0UTM$FINISH; 

current-request-file := input-request-file; 
end procedure ; 



procedure OLD LIST SUB( current-request-f lie ); 

scalar current-request-file; /* The name of the file */ 

/* Retrieve and execute an old list of requests. */ 

scalar input-request-file /* The file containing the requests. */ 
record request; 

record response; /* to a request that has been executed. */ 

/* Determine the new current-request-file to use for this */ 

/* subsession. */ 

perform DETERMINE_INPUT_FILE ( current-request-f lie , 

input-request-f lie ) ; 

Open( input-request-file ) input; 

Read first request from input-request-file; 

/* Determine whether response is to go to CRT, file or both. */ 
perform 0UTM$F0RMAT; 
while more-requests do 

perform EXECUTE( request, response ); 

/* Output the response to CRT, file or CRT&file, as */ 

/* appropriate. */ 

perform 0UTM$RESP0NSE( response ) ; 

Read next request from input-request-file; 
end while; 

perform 0UTM$FINISH; 
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6.9.16 

6.9.17 

6.9.18 



6. 6. 9.1 

6. 6. 9. 2 

6. 6. 9.3 

6 . 6 . 9. 4 



6 . 6 . 9. 5 

6. 6. 9.6 

6 . 6 . 9. 7 

6 . 6 . 9. 8 

6 . 6 . 9. 9 

6.6.9.10 

6.6.9.11 

6.6.9.12 

6.6.9.13 

6.6.9.14 

6.6.9.15 

6.6.9.16 

6.6.9.17 

6.6.9.18 

6.6.9.19 

6.6.9.20 

6 . 6 . 9.21 
6 . 6 . 9.22 

6.6.9.23 

6.6.9.24 



6.7.11.1 

6.7.11.2 

6.7.11 .3 

6.7.11.4 



6.7.11.5 



6.7.11.6 

6.7.11.7 

6.7.11.8 

6.7.11.9 



close( input-request-file ); 
current-request-file := input-request-file; 

end procedure ; 



procedure ENTER_AND_S AVE_RE QUE STS 

( input : request-list-file-name ); 
scalar request-list-file-name; 

/* of file to use to store the requests */ 
record request; 
scalar next-step; 

/* I(nsert), R(etrieve), U(pdate) , D(elete) or F(inish) */ 

next-step := I; 
while next-step ~= F do 
Prompt for next-step; 
case next-step value 

I: /* enter and save the next insert request */ 
perform INSERT_SUB( request ); 

Write request into request-list-file-name ; 

R: /* enter and save the next retrieve request */ 
perform RETRIEVE_SUB( request ) ; 

Write request into request-list-file-name ; 

U: /* enter and save the next update request */ 
perform DELETE SUB( request ); 

Write request into request-list-file-name ; 

D: /* enter and save the next delete request */ 
perform DELETE_SUB( request ) ; 

Write request into request- list-file-name ; 

F: /* Finish entering requests */ 
otherwise : Print error message; 
end case : 
end while ; 
end procedure ; 



procedure DETERMINE_INPUT_FILE( input : current-request-f lie , 
output : input-request-file ) ; 
scalar current-request-file; 
scalar input-request-file; 

/* Determine the input file to be used. It may be either */ 

/* the current-request-file or a different existing */ 

/* request file. */ 

scalar modif y-current-f ile-f lag ; 

/* TRUE - select new input file */ 

if current-request-file is NULL 
then 

Prompt for name of input-request-file; 

Read name of input-request-file; 
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6.7.11 .10 

6.7.11 .11 

6.7.11.12 

6.7.11.13 

6.7.11.14 

6.7.11.15 

6.7.11.16 

6.7.11.17 

6.7.11.18 



else /* Determine if user wants to use the */ 

/* current-request-file or a different old file. */ 
Prompt user to determine modify-current-file-flag; 
Read modify-current-file-flag; 
if modify-current-file-flag 
then 

Prompt for name of input-request-file; 

Read name of input-request-file; 

else 

input-request-file := current-request-file; 



6.7.11.19 end if: 

6.7.11.20 end if: 

6.7.11.21 end procedure ; 



6.7.29.1 

6.7.29.2 



6.7.29.3 



procedure GET_NEW_REQUEST ( output : request ) ; 

record request; /* to be obtained from user */ 

/* Prompts user for information necessary to enter a */ 
/* new request. Returns the request. */ 

scalar request-type; 

/* I(nsert), R(etrieve), U(pdate) or D(elete) */ 



6.7.29.4 

6.7.29.5 

6.7.29.6 

6.7.29.7 

6.7.29.8 

6.7.29.9 

6.7.29.10 

6.7.29.11 

6.7.29.12 



Prompt for .next request-type; 

Read request-type; 

case request-type value 
. I: perform INSERT_SUB( request ) 

U: perform UPDATE_SUB( request ) 
D: perform DELETE_SUB( request ) 
R: perform RETRIEVE_SUB( request 
otherwise : Print error message; 
end case ; 



); 



6.7.29.13 



end procedure ; 



6.8.15.1 procedure DISPLAY( input : requests ); 

/* Display the requests and their numbers at the */ 
/* terminal. */ 

6.8.15.2 array requests( MAX_NUMBER_OF_REQUES IS ); 

/* to be displayed. */ 



6.8.15.3 end procedure ; 
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6. 8. 23.1 procedure EXECUTE( input : request, output : response ); 

/* Ask MDBS to execute this request. Return the response. */ 



6.8.23.2 record request; /* to be executed */ 

6.8.23.3 record response; /* to the execution of the request */ 

6.8.23.4 end procedure ; 



D.2 Part II - The Output Module for Test Execution 



/* (1) Part II - The Output Module for Test Execution */ 
/* (2) Design: OUTM */ 
/* (3) Designer: D.S. Kerr */ 
/* (4) Date: July 8, 1982 */ 
/* */ 
/* (6) Purpose: */ 
/* The following three procedures are used to handle the displaying */ 
/* and/or saving of the responses to the execution of the requests. */ 
/* The default is to display the responses on the CRT. */ 



(8) Procedure Hierarchy for the Module OUTM 



0UTM$ 




FORMAT 




RESPONSE 




FINISH 





(10) Data Structures 



The data structures definitions are included as part of the module and at 
the beginning of each procedure definition in (11) below. 
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(11) Program Specifications 



module OUTM 



programs FORMAT, RESPONSE, FINISH; 
data sets 

/* Variables controlling the output of responses */ 
scalar CRT-output-f lag ; 

/* TRUE if output is to be displayed on CRT. */ 

/* Initial value is TRUE. */ 
scalar file-output-flag; 

/* TRUE if output is to be put into a file. */ 

/* Initial value is FALSE. */ 

/* CRT-output-f lag and/or file-output-flag must be TRUE */ 
scalar response-file-name; 

/* the name of the file if output is to be made */ 

/* to a file. */ 

end module ; 



1 . 



2 . 



3. 

4. 

5. 

6 . 

7. 

8 . 

9. 

10 . 
11 . 
12 . 

13. 

14. 

15. 

16. 

17. 

18. 



procedure FORMAT; 

/* Determines what form of output is to be used. Opens */ 
/* response file, if appropriate. */ 

scalar change- in-output ; /* flag: TRUE - prompt user for */ 
/* how to change output. */ 



Prompt for change- in-output ; 

Read change- in-output ; 
if change- in-output 
then begin 

Prompt for output form: CRT, file, both CRT&file; 
Read output form; 

Set CRT-output-f lag; 

Set file-output-flag; 
if file-output-flag 
then 

Prompt for response-file name; 

Read response-file-name; 

Open response-file-name; 

end if ; 

end if : 

end procedure ; 



m<tinvDr^ooiTiHH h m <t 
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procedure RESPONSE( input : response ); 

/* Outputs the response. */ 

record response; /* to be output */ 

if CRT-output-f lag 
then 

Print response on CRT; 

end if : 

if file-output-flag 
then 

Write response in file( response-f lie-name ); 

end if ; 

end procedure ; 



procedure FINISH; 

/* Carries out whatever processing is needed when a subsession */ 
/* is completed, closes response-file-name if appropriate. */ 

if file-output-flag 
then 

close( response-file-name ); 

end if ; 

end procedure ; 
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